# DNN-Life: An Energy-Efficient Aging Mitigation Framework for Improving the
Lifetime of On-Chip Weight Memories in Deep Neural Network Hardware
Architectures

Muhammad Abdullah Hanif1, Muhammad Shafique2 1Faculty of Informatics,
Technische Universität Wien (TU Wien), Vienna, Austria
2Division of Engineering, New York University Abu Dhabi (NYUAD), Abu Dhabi,
United Arab Emirates
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Negative Biased Temperature Instability (NBTI)-induced aging is one of the
critical reliability threats in nano-scale devices. This paper makes the first
attempt to study the NBTI aging in the on-chip weight memories of deep neural
network (DNN) hardware accelerators, subjected to complex DNN workloads. We
propose DNN-Life, a specialized aging analysis and mitigation framework for
DNNs, which jointly exploits hardware- and software-level knowledge to improve
the lifetime of a DNN weight memory with reduced energy overhead. At the
software-level, we analyze the effects of different DNN quantization methods
on the distribution of the bits of weight values. Based on the insights gained
from this analysis, we propose a micro-architecture that employs low-cost
memory-write (and read) transducers to achieve an optimal duty-cycle at run
time in the weight memory cells, thereby balancing their aging. As a result,
our DNN-Life framework enables efficient aging mitigation of weight memory of
the given DNN hardware at minimal energy overhead during the inference
process.

##  I Introduction

DNN accelerators have already become an essential part of various machine
learning systems [1][2]. DNNs usually require a large number of parameters to
offer high accuracy, which comes at the cost of high memory requirements; see
Fig. 1a. Dedicated memory hierarchies are designed to tradeoff between the
low-cost storage offered by the off-chip DRAMs and the energy-/performance-
efficient access offered by the on-chip SRAMs [1]; see Fig. 1b for access
energy. This has led to an increasing trend towards the use of larger on-chip
memory in the state-of-the-art DNN accelerators [3][4], with the recent wafer-
scale chips having up to 18 GBs of on-chip memory [5]. However, due to
continuous technology scaling, the on-chip SRAM-based memories are becoming
increasingly vulnerable to different reliability threats, for example, soft
errors and aging [6][7][8]. Studies have shown that even a single fault in
weights of critical neurons can result in significant degradation of
application-level accuracy [9]. State-of-the-art works have focused on
analyzing and mitigating the effects of faults in DNN accelerators w.r.t. DNN
accuracy [10]. However, to the best of our knowledge, no prior works have
analyzed and optimized the aging of the on-chip weight memories of DNN
accelerators, especially when considering diverse dataflows of different DNNs
and the impact of different types of quantizations on the weight
distributions.

Aging due to NBTI: In PMOS transistors when a negative gate-to-source voltage
is applied, it can break-down the Si-H bond at the oxide-interface, thereby
causing a gradual increase in the threshold voltage ($V_{th}$) over the device
lifetime, which results in poor drive current and a reduction in the noise
margin [11]111A similar phenomenon called PBTI happens in NMOS transistors,
though NBTI has been considered relatively more serious compared to PBTI [6]..
To overcome this $V_{th}$ shift, the operating frequency of the device has to
be reduced by more than 20% over its entire lifetime [12]. However, due to
strict performance and energy constraints (specifically for embedded
applications), the $V_{th}$ shift cannot be addressed just by design-time
delay margins or adaptive operating frequency adjustments [13], as this leads
to a significant loss in the system’s performance and energy efficiency.
Therefore, in traditional computing systems, alternate opportunities have to
be exploited to overcome this challenge [12]. One such opportunity lies in the
fact that the NBTI aging phenomenon is partially reversed by removing the
stress.

Figure 1: (a) Accuracy and size comparison of few of the state-of-the-art DNNs
(b) Access energy comparison of SRAM with DRAM (data source: [1]).

NBTI Aging of On-chip Memories: On-chip memories are typically built using
6T-SRAM cells to achieve high area and power efficiency. A 6T-cell is composed
of two inverters coupled with two access transistors (see Fig. 2a). The
inverters store complementary values to store a single bit. Each inverter has
a PMOS transistor and an NMOS transistor. Depending on whether the cell is
storing ‘0’ or ‘1’, one of the PMOS transistors is always under stress, when
the transistor is on. As aging of a cell is defined by its most-aged
transistor, the lowest aging is achieved when both the PMOS transistors
receive on-average the same amount of stress over the entire lifetime of the
device, i.e., the percentage of the entire lifetime for which the cell stores
a ‘1’ (duty-cycle) is 50%, as shown in Fig. 2b. Note that NBTI aging strongly
depends on average long-term stress and weakly on short-term statistics [14].
Therefore, the key challenge in aging mitigation of on-chip memories is to
balance their duty-cycle over the entire lifetime without affecting system-
level performance.

Figure 2: (a) A 6T-SRAM Cell; and (b) its SNM degradation after 7 years [15]

State-of-the-art techniques and their limitations: Various techniques have
been proposed at circuit-level and at architecture-level. At circuit-level,
the structure of SRAM-cells is modified to reduce the aging rate [16][13]. For
example, Ricketts et al. [16] proposed an asymmetric SRAM structure for
workloads having biased bit distribution, but due to their high data
dependence, they are applicable only in specific scenarios. Recovery boosting
through dedicated recovery accelerating circuit is another method for
enhancing the lifetime of the SRAM cells [17], but it increases power/energy
consumption due to additional transistors per cell, and therefore cannot be
used in energy-constrained large-sized memories [18]. At architecture-level,
periodic inversion of data is used to reduce the aging rate of on-chip caches
[19]. However, it cannot guarantee optimal duty-cycle, specifically in cases
where the same data is periodically reused, e.g., in DNN-based systems where
the same set of parameters are reused for processing each input sample.
Calimera et al. in [20] improved recovery of unutilized portions of memory,
but at high area & energy cost of expensive online monitoring. The technique
also suffers from serious performance degradation in dynamic workload
scenarios. Another set of techniques uses bit rotations to cater NBTI aging in
registers [15], but they work only in cases where the overall distribution of
bits is relatively balanced. Moreover, they use barrel shifters that incur
high area and power overheads. The work in [21] proposed a configurable micro-
architecture for reducing aging rate of video memories, but only works for
streaming video applications.

In summary, the state-of-the-art techniques either incur high overheads in
terms of area and power/energy or rely on certain specific workloads, but
cannot be employed in DNN accelerators due to the unique properties of DNN
hardware and workloads, as we will illustrate later in this paper.

Additional Challenges from the Deep Learning Perspective: The dataflow (i.e.,
computation scheduling) for a given DNN on a specific hardware is defined as
per the DNN architecture and the hardware implementation to achieve maximum
energy-/performance-efficiency. Altering the dataflow to balance the duty-
cycle in on-chip SRAM cells can result in significant degradation of system-
level efficiency. Therefore, an aging mitigation technique that does not
require any alteration to the dataflow or the mapping of the data in on-chip
SRAM is desired.

Our Novel Contributions: Towards this, we propose DNN-Life, an aging analysis
and mitigation framework for on-chip memories of DNN hardware (see Fig. 3).
Our framework employs two key features:

  1. 1.

Aging Analysis [Section III]: We analyze the impact of using different data
representation formats and quantization methods for weights of a DNN on the
probability distribution of weight-bits, as this can provide useful insights
for designing an effective and low-overhead aging mitigation technique.

  2. 2.

Aging Mitigation [Section IV]: We propose a scheme and supporting micro-
architecture for mitigating the NBTI-aging of 6T-SRAM-based on-chip weight
memory of DNN accelerators with minimal energy overhead. Noteworthy, our
scheme does not require any alteration to the dataflow of DNN inference or on-
chip data mapping, and thereby maintains the energy and performance benefits
of the system. The micro-architectural extensions for aging mitigation are
integrated in the DNN accelerator before and after the on-chip weight memory
in the form of aging-aware write and read transducers, as shown in Fig. 4.

Figure 3: Overview of the design-time and run-time steps involved in our DNN-
Life framework. Our novel contributions are highlighted in colored boxes.

##  II Overview of Our DNN-Life Framework

Figure 4: (a) Architecture of the baseline DNN accelerator. The highlighted
boxes, i.e., Write Data Encoder (WDE), Read Data Decoder (RDD) and Aging
Controller, are the proposed modules for mitigating NBTI aging of weight
memory. (b) A detailed view of the processing array and the accumulation unit.

In this work, we propose DNN-Life, a novel aging analysis and mitigation
framework for weight memories of DNN hardware accelerators. It employs a low-
cost data encoding scheme that accounts for diverse DNN workloads to adapt
over time to balance the duty-cycle in each on-chip weight memory cell to
alleviate the NBTI-aging effects. Towards this, the two key features of our
framework are:

  1. 1.

Analysis: We analyze the probability distribution of weight-bits of different
pre-trained DNNs to find key insights that help in developing a low-cost
aging-mitigation scheme. To consider the variations in the distribution across
number representation formats and the methods used to transform the weights to
those formats, we consider different number representation formats and
different commonly used conversion methods. The detailed analysis and insights
are presented in Section III.

  2. 2.

Architecture: Based on the gathered insights, we design a data encoding module
and an aging controller. The encoder is responsible for encoding the weights
before writing the values to the weight memory, and the aging controller is
responsible for generating encoding information required to encode the data
such that the duty-cycle is balanced. The encoding information is then stored
to be used by the corresponding decoder module. The data encoder is deployed
inside the DNN hardware accelerator right before the weight memory, and the
corresponding decoder is installed after the memory, to decode the weights
before passing them for computations. The integration of the encoder and the
decoder modules in a DNN accelerator is illustrated in Fig. 4a. The details of
the micro-architecture are presented in Section IV.

###  II-A DNN Hardware Architecture

Our DNN hardware architecture is based on well-established DNN accelerator
models, such as [22] for dense DNNs. Our accelerator is composed of an
Activation Buffer, a Weight Buffer, a Processing Array, and an Accumulation
Unit; see Fig. 4a. Our proposed weight-memory aging mitigation modules
integrated in the architecture are also shown in the figure (see details in
Section IV). The activation and weight buffers provide intermediate storage
for the activations and weights, respectively, to reduce the costly off-chip
memory accesses. The buffers provide data to the processing array for
performing the computations. For this work, we assume a memory hierarchy
similar to Bit-Tactical [22], DaDianNao [3] and TPU [4], according to which:
1) the activation buffer is large enough to store the activations of a single
layer of a DNN; 2) the activation memory can provide $N$ number of activation
values to the processing array at a time; and 3) the weight memory can provide
$f\times N$ weights to the processing array simultaneously. The processing
array (see Fig. 4b) is composed of $f$ number of Processing Elements (PEs)
that share the activations, and therefore can perform $N$ number of
multiplications for $f$ different filters at the same time. Each PE has an
adder tree to compute the sum of the multiplications. The computed sum is
passed to the accumulation unit where it is added with the corresponding
partial sums to generate the output activation value. Note, as the filters can
be significantly large, the computation of each output activation can take
several cycles, depending on the filter size.

Figure 5: Division of filters of a CONV layer of a DNN into smaller blocks
that can be accommodated in the on-chip weight memory. Different colors
correspond to different sets of filters/blocks. The gray colored boxes define
one block of $r\times c\times ch\times f$ size. The steps show the sequence in
which the blocks are moved to the on-chip fabric for scheduling their
computations.

###  II-B Dataflow in the DNN Accelerator

To perform the computations of a DNN layer using the above accelerator, the
weights have to be partitioned into blocks that can be accommodated in the on-
chip memory. The goal of partitioning is to maximize the use of available PEs.
The input/output feature maps and the filters/neurons all are divided into so-
called tiles, depending on the available on-chip storage for the corresponding
data type. Works like SmartShuttle [23] provide methods to find an optimal
tiling configuration and computation scheduling policy for a layer of a DNN
for a given memory hierarchy.

Fig. 5 illustrates the policy that we employ for partitioning the filters of a
CONV layer. Note, we support the well-established tiling technique so that we
can demonstrate that our technique can benefit a wide-range of existing DNN
hardware accelerators. The figure also illustrates the sequence in which the
blocks are moved to the on-chip weight memory and the corresponding
computations are scheduled. The filters are first divided into sets, where
each set contains f number of filters. Note, f is mainly defined based on the
number of filters that the hardware accelerator can process in parallel.
Afterwards, a chunk of data (grey boxes in Fig. 5) from a set is selected to
be moved to the on-chip memory. The selected chunk contains a block of data of
size $r\times c\times ch$ from the same location of each filter in the set.
The sequence in which the grey boxes are traversed in the filters defines rest
of the dataflow. The used sequence is shown as steps in Fig. 5.

##  III Analysis of the Distribution of Weight-Bits for Different DNNs & their
Impact on Duty-Cycle

Before presenting the design of the proposed aging mitigation modules in
Section IV, here we first present an analysis which highlights the rationale
behind the proposed design.

###  III-A Analyzing the Distribution of Weight-Bits

For this analysis, we consider the AlexNet and the VGG-16 networks, trained on
the ImageNet dataset. As different data representations for weights, we
consider 32-bit floating point representation (IEEE 754 standard) and 8-bit
integer format achieved using range-linear symmetric and asymmetric
quantization techniques [24]. Fig. 6 illustrates the ratio of observing a ‘1’
to the total number of observations (which corresponds to probability of
observing a ‘1’) at each bit-location of a word for all three data
representation formats for both the networks. By analyzing the distributions,
the following key observations are made:

  1. 1.

The probability of getting a ‘1’ value at a particular bit-location of a
randomly selected weight depends on the network, the data representation
format, and the method used to transform the data to the particular data
representation format. For example, the probability of getting a ‘1’ at a
particular bit-location in symmetric 8-bit representation is almost the same
across bit-locations within a network for both the considered DNNs, however,
it varies across networks. Similarly, the probability of getting a ‘1’ at the
lower bit-locations in 32-bit floating-point representation is around 0.5,
however, the distribution of bits at higher bit-locations varies across bit-
locations as well as across DNNs.

  2. 2.

Representation of weights using a specific format cannot guarantee a
distribution that offers 0.5 probability at each bit-location, i.e., a
distribution that can potentially lead to a balanced duty-cycle. For example,
out of all the studied cases, only the distribution of the AlexNet when
represented using 8-bit integer format achieved using symmetric range-linear
quantization offers close to 0.5 probability for all the bit-locations.

  3. 3.

The average probability of getting a ‘1’ across bit-locations in a specific
format is also not guaranteed to be equal to 0.5. For example, see the
distributions of 8-bit asymmetrically quantized DNNs. Therefore, barrel
shifter-based balancing techniques would not produce desirable results in such
cases.

Figure 6: Distribution of bits of weights of different different DNNs when
represented in different data representation formats. Symmetric and asymmetric
represent which post-training quantization method is used to transform the
data for the corresponding distribution.

###  III-B A Probabilistic Model-based Analysis for Aging of 6T-SRAM
On-chip Weight Memory of a DNN Accelerator

In the following, we develop a probabilistic model to analyze the
effectiveness of different aging mitigation techniques.

####  III-B1 Probabilistic Model

Assume the on-chip memory of a given DNN accelerator is composed of $I\times
J$ cells. For mapping the weights of a DNN onto the memory, we assume: (a) the
same dataflow as presented in Fig. 5; (b) each block of weights is kept in the
on-chip memory for equal amount of time, and it is fetched only once during a
single inference (similar to the dataflow for the DNN accelerator proposed in
[22]); (c) each block of data mapped onto the on-chip memory fits perfectly to
it. Based on the aforementioned conditions and the given DNN size, we can
divide the DNN into $K$ blocks that translates to $K$ number of data mappings
onto the on-chip weight memory. Now, if the same DNN is used repeatedly for
inferencing with the same dataflow, a single on-chip memory cell is mapped
with only $K$ different bits. If the probability of getting a ‘1’ for all the
bits is given by $\rho$, the probability of getting a duty-cycle less than and
equal to $b/K$, or greater than and equal to $1-b/K$, can be computed using
the following equation, except when $b/K=0.5$, where the probability is 1.

$\footnotesize
P_{b/K}=\sum_{i=0}^{b}\binom{K}{i}\rho^{i}\times(1-\rho)^{K-i}+\sum_{i=K-b}^{K}\binom{K}{i}\rho^{i}\times(1-\rho)^{K-i}$
(1)

Here, $b$ is an arbitrary variable with the range from $0$ to $\lfloor
K/2\rfloor$. Note that we combine (i) the cases in which duty-cycle is less
than and equal to $b/K$ and (ii) the cases in which duty-cycle is greater than
and equal to $1-b/K$, because in a symmetric 6T-SRAM cell both the cases cause
the same level of stress in one of the two PMOS transistors. Assuming the
above computed probability to be the same for all the cells of the on-chip
memory, the probability of at least $n$ number of cells (out of $I\times J$)
experiencing duty-cycle less than and equal to $b/K$, or greater than and
equal to $1-b/K$ can be computed using the following equation.

$\footnotesize P_{n}=\sum_{i=n}^{I\times J}\binom{I\times
J}{i}P_{b}^{i}\times(1-P_{b})^{I\times J-i}$ (2)

####  III-B2 An Example Case-Study

Let us consider a scenario where $K=20$ and $\rho=0.5$ (i.e., the best-case
with balanced bit distribution), and $I\times J=8192$. Fig. 7a shows the
probability for each possible value of $b$ computed using Eq. 1. Note, even
for $b/K=0.3$, the probability is over 0.1, i.e., more than 10% of the cells
are expected to experience a duty-cycle of less than 0.3, or greater than 0.7.

Figure 7: Probability of occurrence of $b/K\geq$ duty-cycle $\geq 1-b/K$ when
(a) $K=20$, and (b) $K=160$

Now, if we employ a given aging mitigation technique that offers upto 7 shifts
to increase the number of different bits that are mapped to a single cell, we
can theoretically increase the value of $K$ to 160, assuming the bits to be
independent from each other and the ideal shifting policy. Putting $K=160$ in
the above mentioned example, Fig. 7b shows the probabilities for different
$b/K$ values. As can be seen from Fig. 7b, the probabilities at lower $b/K$
values have dropped significantly. The above analysis implies that by
significantly increasing $K$ and having $\rho=0.5$, we can achieve close to
ideal duty-cycle for all the cells.

Now, instead of a barrel shifter, if we employ an inversion-based duty-cycle
balancing technique where every other write to the same location is inverted,
for the given scenario, the value of $K$ remains the same, as it is even.
Moreover, as $\rho$ is defined to be 0.5, the inversion-based policy has no
impact on $\rho$ either. Therefore, we get the same probabilities as presented
in Fig. 7a. However, note that the inversion-based policy is mainly useful for
achieving $\rho=0.5$ in cases where the distribution of bits is biased either
towards ‘0’ or ‘1’.

###  III-C Challenges in Designing an Efficient Aging Mitigation System

Based on the above analysis, we outline the following key challenges in
designing a generic aging mitigating system.

  1. 1.

The probability of occurrence of non-ideal duty-cycle is considerable even
with the state-of-the-art fixed aging mitigation techniques. Therefore, a more
robust method has to be designed by exploiting the fact that NBTI-aging is
more dependent on the average duty-cycle over the lifetime of the device [14].

  2. 2.

The distribution of bits and the duty-cycle is significantly affected by the
datatype used for representing the weights. Therefore, the mitigation
technique should be generic and independent of the datatype used so that it is
beneficial for various DNN accelerators.

Moreover, in practical scenarios, each layer of a DNN can have a different
size. Therefore, each layer can take different amount of time for processing
that can vary significantly across layers. Also, different DNNs can have
different number of layers. Therefore, a method that keeps track of all these
factors at a fine granularity can help in significantly reducing the aging
rates. However, such methods are super costly. This makes it very challenging
to develop a generic method that offers effective aging mitigation at
reasonable overheads.

##  IV A Micro-architecture for Mitigating Aging of the On-Chip Weight Memory
of DNN Accelerators

To address the above challenges, we propose a Write Data Encoder (WDE) for
encoding the weights before writing them to the on-chip weight memory, and a
Read Data Decoder (RDD) which performs the inverse function of the WDE while
reading the data from the on-chip memory and before passing it to the
processing array. The integration of the proposed modules in the DNN
accelerator is shown in Fig. 4a. Moreover, we propose an aging mitigation
controller which generates the control signals (metadata) for the write (and
read) transducer. The proposed micro-architectures of the WDE and the aging
mitigation controller is shown in Fig. 8.

Figure 8: Proposed micro-architecture for effective aging mitigation of
6T-SRAM weight memory of DNN accelerators.

Write Data Encoder (WDE): It leverages the inversion logic that besides its
low-overhead222low overhead compared to other techniques such as shifting,
which requires costly barrel shifters (as shown later in Section V), also
enables to perfectly balance out the distribution of bits in the cells of the
memory when the distribution is originally biased towards either ‘0’ or ‘1’,
as highlighted in Section III. The inversion logic in the proposed micro-
architecture is implemented using XOR gates as they allow the aging mitigation
controller to enable or disable it using just a 1-bit enable ($E$) signal.
Another key advantage of this design is that the micro-architecture of the RDD
is the same as WDE, where the same $E$ signal (metadata) that is used to
encode the weights is used (at a later point in time) for decoding them before
passing them to the processing array. Moreover, the proposed WDE and RDD
modules are highly scalable, as increasing the width of the modules require
only a linear increase in the number of XOR gates. Therefore, the widths of
these modules can be defined directly based on the DNN accelerator
configuration without affecting the energy-efficiency of the system.

Aging Mitigation Controller: The controller is the core part of the proposed
micro-architecture, as it is responsible for generating the enable signal
($E$) that enables/disables the inversion logic in WDE. The design is based on
the observations made in Section III that the higher the number of different
bits to be written on an SRAM cell during its lifetime (i.e., $K$ in Eq. 1)
that are generated from a uniform distribution the lower the chances of
observing a deviation in its duty-cycle from 0.5 (see Figs. 7), i.e., the
ideal point shown in Fig. 2b. Therefore, to increase the number of different
bits to be written on an SRAM cell, we employ a True Random Bit Generator
(TRBG) to generate the enable signal and decide whether the upcoming data
should be written with or without inversion in the memory cell. TRBG adds the
sense of randomness in the bits to be written in the memory and thereby leads
to larger $K$ value and lower aging.

Note in practical scenarios, the output of TRBGs can be biased towards either
‘0’ or ‘1’, which can eventually affect the duty-cycle. Therefore, to mitigate
this, we periodically invert the output of the TRBG after a defined number of
iterations with the help of an $M$-bit register before using it as the enable
signal, which balances the bias.

Figure 9: SNM degradation of 6T-SRAM on-chip weight memory cells of the
baseline DNN accelerator when used for performing inferences only using the
AlexNet network. Each bar graph shows the percentage of the number of cells
(Y-axis) experiencing different level of SNM degradation (X-axis). Figure 10:
Overall experimental setup used for evaluation.

##  V Results and Discussion

###  V-A Experimental Setup

Fig. 10 illustrates the overall experimental setup used for evaluation. The
setup consists of hardware synthesis for estimating the power, area and delay
characteristics of the proposed modules, and simulations for aging estimation
of the 6T-SRAM on-chip weight memory of different DNN hardware accelerators.
For hardware synthesis, we implemented different aging balancing circuits and
our DNN-Life architecture in Verilog. The circuits are synthesized for the
TSMC 65nm technology using Cadence Genus.

For aging estimation, we use Static Noise Margin (SNM) to quantify the NBTI-
aging of 6T-SRAM cells, similar to [21][25]. The SNM defines the tolerance to
noise that directly affects the read stability of a cell [26], i.e., if the
SNM of a cell is low, the cell is highly susceptible to read failures. As per
[15][21][25], SNM mainly depends on the duty-cycle over the entire lifetime of
the cell, and the least SNM degradation is achieved at 50% duty-cycle. To
obtain SNM results, we employ a similar device aging model as used in state-
of-the-art studies like [21][25]. However, due to its duty-cycle optimization
focus, our proposed technique is orthogonal to the given device aging models,
and other device-level models can easily be integrated in our framework. Based
on the models, the SNM degradation of a 6T-SRAM cell can be computed using the
duty-cycle. From the analysis, the best SNM degradation for 6T-SRAM cell after
7 years is 10.82% (at 50% duty-cycle), and the worst is 26.12% (at 0% and 100%
duty-cycle).

For large-scale simulations, we integrated the output of these models into a
memory simulator of the baseline DNN hardware (described in Section II-A). The
simulator takes the DNN hardware configuration, dataflow, pre-trained DNN
architecture and test samples as inputs. We also built a memory simulator for
a TPU-like hardware architecture [4] to validate the proposed aging-mitigation
technique across DNN hardware accelerators. The hardware configurations used
for the evaluation are presented in Table I. The DNNs used are the AlexNet and
the VGG-16 with the ImageNet dataset and a custom network with MNIST dataset.
The custom network is composed of two CONV layers and two FC layers, i.e.,
CONV(16,1,5,5), CONV(50,16,5,5), FC(256,800) and FC(10,256). For each setting
the duty-cycles are estimated based on the values observed in 100 inferences.
The bias balancing register is defined to be a 4-bit register (i.e., M=4), for
all the corresponding cases.

TABLE I: Hardware configurations and settings used in evaluation

|  | Baseline Accelerator (Section II-A)
---
TPU-like NPU [4]
| Weight
---
memory size
| 512KB
---
256KB
| Activation
---
memory size
4MB | 24MB
| PE array size
---
| 8 PEs (1 PE = 8 Multipliers)
---
| 256 x 256 PEs (1 PE = 1 MAC)
---
Networks | AlexNet |  | AlexNet, VGG-16 and Custom
---

###  V-B Aging Estimation Results and Comparisons

In this subsection, we analyze the impact of using different aging mitigation
policies on the SNM degradation of the 6T-SRAM on-chip weight memory cells
after 7 years. We mainly considered four different policies: (1) No aging
mitigation, (2) Inversion-based, (3) Barrel shifter-based, and (4) DNN-Life.
For the proposed DNN-Life, we consider three different cases: (i) TRBG is not
biased and it generates 0s and 1s with equal probability (referred in the
results as Bias=0.5); (ii) TRBG is biased and it generates 1s with 0.7
probability, and the aging controller does not have a bias balancing register
(referred in the results as without bias balancing with Bias=0.7); and (iii)
TRBG is biased and it generates 1s with 0.7 probability and the aging
controller has a 4-bit bias balancing register (referred in the results as
with bias balancing with Bias=0.7).

Moreover, we performed experiments considering three different data
representation formats for weights: (1) 32-bit floating point format; (2)
8-bit integer format when weights are quantized using symmetric quantization
method; and (3) 8-bit integer format when weights are quantized using
asymmetric quantization method.

Fig. 9 shows the distributions of SNM degradation in the memory cells obtained
using different aging mitigation policies and a pre-trained AlexNet model. The
Y-axis of each bar graph shows the percentage of the number of cells and the
X-axis of each shows SNM degradation levels. Note that, for these experiments,
we assumed the baseline DNN accelerator configuration presented in Table I and
the dataflow shown in Fig. 5 with $f=8$. Also, we assumed that only a single
DNN (i.e., the AlexNet) is used for data inference throughout the lifetime of
the device. As can be seen in the figure, the inversion-based and barrel
shifter-based aging balancing reduce the SNM degradation of the SRAM cells,
however, they do not offer minimum SNM degradation (see 2 and 3 in comparison
with 1 in Fig. 9). This behavior is observed to be consistent across all the
data representation formats (see 2 till 7 in comparison with their respective
without aging mitigation graphs in Fig. 9). Specifically, the inversion-based
aging balancing offers sub-optimal aging mitigation in case of the 32-bit
floating point format (see 2 in Fig. 9), where most of the cells experience
around 10.8% SNM degradation (see a in Fig. 9). However, this is not the ideal
scenario as there are 4% cells that experience highest level of SNM
degradation (see b in Fig. 9) and a few that experience moderate level of SNM
degradation (see c in Fig. 9). Now, if we analyze the results of the proposed
DNN-Life with bias balancing, it offers maximum aging-mitigation (i.e., all
the cells experience around 10.8% SNM degradation) in all the cases (see 8, 9
and 10 in Fig. 9).

Impact of biased TRBG on aging balancing of 6T-SRAM on-chip weight memory:
Fig. 9 also illustrates the impact of using proposed design without bias
correction when the duty-cycle of TRBG is 0.7. As can be seen in the figure,
for all the data representation formats, having biased TRBG and no bias
correction leads to less reduction in SNM degradation of the 6T-SRAM cells
(e.g., see 11 in comparison with 8 in Fig. 9). This behavior is consistent
across all the data representation formats.

Figure 11: SNM degradation of 6T-SRAM on-chip weight memory cells of a TPU-
like NPU when used for performing inferences using the AlexNet, the VGG-16 and
the custom DNN, individually. The networks are quantized to 8-bit format using
symmetric range-linear quantization method.

Impact across different hardware accelerators: Fig. 11 shows the impact of
using the proposed aging-mitigation technique for a TPU-like [4] Neural
Processing Unit (NPU) architecture that has an on-chip weight FIFO which is
four tiles deep, where one tile is equivalent to weights for $256\times 256$
PEs. Each PE has a single MAC unit that can perform 8-bit multiplication. For
our implementation, we assumed the weight FIFO to be a circular buffer-based
design. We performed analysis using the three different networks mentioned
earlier. All the DNNs are quantized to 8-bits using post-training symmetric
quantization. Considering the dataflow of the NPU, the parameter $f$ was set
to 256. As can be seen in Fig. 11, the inversion-based aging mitigation policy
offers optimal results for the AlexNet and the VGG-16 networks (see 1 and 2 in
Fig. 11). However, when used for the custom DNN, almost all the memory cells
experience significant SNM degradation (see 3 in Fig. 11). The barrel shifter-
based approach also offer sub-optimal results (see 4 till 6 in Fig. 11).
However, the proposed DNN-Life with bias balancing offers maximum aging
mitigation (see 7 till 9 in Fig. 11). This shows that DNN-Life can be used for
a wide range of DNN accelerators.

###  V-C Area and Power Results

The area, power and delay characteristics of three different WDEs composed of
different aging balancing units are shown in Table II. All three WDEs are
designed for 64 bit-width. The barrel shifter-based WDE consumes the most
amount of area and power. The proposed design consumes slightly more power and
area as compared to the inversion-based WDE. However, as shown in the previous
subsection, it offers best aging-mitigation in all the possible scenarios
regardless of the size of the given DNN, the data representation format and
the on-chip weight memory size. Note that, at hardware level, we realized TRBG
using a 5-stage ring oscillator.

TABLE II: Hardware results of different Write Data Encoders (WDEs)

| Delay [ps] | Power [nW] |  | Area [cell area]
---
Barrel Shifter based WDE | 977.7 | 345190 | 9035
Inversion based WDE | 811.6 | 10716 | 195
| Proposed WDE with Aging
---
Mitigation Controller
581.8 | 13747 | 295

##  VI Conclusion

In this paper, we proposed DNN-Life, an aging-mitigation framework that
employs read and write transducers to reduce NBTI-induced aging of 6T-SRAM on-
chip weight memory in DNN hardware accelerators. We analyzed different DNN
data representation formats at the software-level and their potential for
balancing the duty-cycle in SRAM cells. Based on the analysis, we proposed a
micro-architecture that makes use of a True Random Bit Generator (TRBG) to
ensure optimal duty-cycle at runtime, thereby balancing the aging of
complimentary parts in 6T-SRAM cells of the weight memory. As a result, our
DNN-Life enables efficient aging mitigation of weight memory of a given DNN
hardware with minimal energy overhead.

## Acknowledgment

This work is partially supported by Intel Corporation through Gift funding for
the project ”Cost-Effective Dependability for Deep Neural Networks and Spiking
Neural Networks.”

## References

  * [1] V. Sze et al., “Efficient processing of deep neural networks: A tutorial and survey,” _Proceedings of IEEE_ , vol. 105, no. 12, pp. 2295–2329, 2017.
  * [2] M. Capra et al., “Hardware and software optimizations for accelerating deep neural networks: Survey of current trends, challenges, and the road ahead,” _IEEE Access_ , 2020.
  * [3] Y. Chen et al., “Dadiannao: A machine-learning supercomputer,” in _IEEE/ACM MICRO Symposium_, 2014, pp. 609–622.
  * [4] N. P. Jouppi et al., “In-datacenter performance analysis of a tensor processing unit,” in _ACM/IEEE ISCA_, 2017, pp. 1–12.
  * [5] P. McLellan. (2019) Hot chips: The biggest chip in the world. Accessed: 2019-09-10. [Online]. Available: https://community.cadence.com/cadence_blogs_8/b/breakfast-bytes/posts/the-biggest-chip-in-the-world
  * [6] J. Henkel et al., “Reliable on-chip systems in the nano-era: Lessons learnt and future trends,” in _ACM/ESDA/IEEE DAC_, 2013, p. 99.
  * [7] M. Shafique et al., “Robust machine learning systems: Challenges, current trends, perspectives, and the road ahead,” _IEEE Design & Test_, vol. 37, no. 2, pp. 30–57, 2020.
  * [8] J. Henkel et al., “Thermal management for dependable on-chip systems,” in _IEEE ASP-DAC_, 2013, pp. 113–118.
  * [9] M. A. Hanif et al., “Robust machine learning systems: Reliability and security for deep neural networks,” in _IEEE IOLTS_ , 2018, pp. 257–260.
  * [10] S. Kim et al., “Matic: Learning around errors for efficient low-voltage neural network accelerators,” in _IEEE DATE_, 2018, pp. 1–6.
  * [11] K. Kang et al., “Nbti induced performance degradation in logic and memory circuits: How effectively can we approach a reliability solution?” in _IEEE ASP-DAC_, 2008, pp. 726–731.
  * [12] D. Gnad et al., “Hayat: Harnessing dark silicon and variability for aging deceleration and balancing,” in _2015 52nd ACM/EDAC/IEEE DAC_ , 2015.
  * [13] J. Shin et al., “A proactive wearout recovery approach for exploiting microarchitectural redundancy to extend cache sram lifetime,” in _ACM/IEEE SIGARCH Computer Arch. News_ , vol. 36, no. 3, 2008, pp. 353–362.
  * [14] J. Abella et al., “Penelope: The nbti-aware processor,” in _IEEE/ACM MICRO Symposium_. IEEE Computer Society, 2007, pp. 85–96.
  * [15] S. Kothawade et al., “Analysis and mitigation of nbti aging in register file: An end-to-end approach,” in _IEEE ISQED_, 2011, pp. 1–7.
  * [16] A. Ricketts et al., “Investigating the impact of nbti on different power saving cache strategies,” in _IEEE DATE_, 2010, pp. 592–597.
  * [17] T. Siddiqua et al., “Enhancing nbti recovery in sram arrays through recovery boosting,” _IEEE TVLSI_, vol. 20, no. 4, pp. 616–629, 2011.
  * [18] B. Zatt et al., “A low-power memory architecture with application-aware power management for motion disparity estimation in multiview video coding,” in _IEEE/ACM ICCAD_, 2011, pp. 40–47.
  * [19] T. Jin et al., “Aging-aware instruction cache design by duty cycle balancing,” in _IEEE IVLSI_, 2012, pp. 195–200.
  * [20] A. Calimera et al., “Partitioned cache architectures for reduced nbti-induced aging,” in _IEEE DATE_, 2011, pp. 1–6.
  * [21] M. Shafique et al., “Enaam: Energy-efficient anti-aging for on-chip video memories,” in _ACM/IEEE DAC_, 2015, pp. 101:1–101:6.
  * [22] A. Delmas et al., “Bit-tactical: Exploiting ineffectual computations in convolutional neural networks: Which, why, and how,” _preprint arXiv:1803.03688_ , 2018.
  * [23] J. Li et al., “Smartshuttle: Optimizing off-chip memory accesses for deep learning accelerators,” in _IEEE DATE_, 2018, pp. 343–348.
  * [24] D. Lin et al., “Fixed point quantization of deep convolutional networks,” in _ICML_ , 2016, pp. 2849–2858.
  * [25] M. Shafique et al., “Content-aware low-power configurable aging mitigation for sram memories,” _IEEE Transactions on Computers_ , vol. 65, no. 12, pp. 3617–3630, 2016.
  * [26] K. Agarwal et al., “Statistical analysis of sram cell stability,” in _ACM/IEEE DAC_, 2006, pp. 57–62.


# Mean dimension of product spaces:
A fundamental formula

Lei Jin  Lei Jin: School of Mathematics, Sun Yat-sen University, Guangzhou,
China<EMAIL_ADDRESS>and  Yixiao Qiao  Yixiao Qiao (Corresponding
author): School of Mathematics and Statistics, Guangdong University of
Technology, Guangzhou, China<EMAIL_ADDRESS>
###### Abstract.

Mean dimension is a topological invariant of dynamical systems, which
originates with Mikhail Gromov in 1999 and which was studied with deep
applications around 2000 by Elon Lindenstrauss and Benjamin Weiss within the
framework of amenable group actions. Let a countable discrete amenable group
$G$ act continuously on compact metrizable spaces $X$ and $Y$. Consider the
product action of $G$ on the product space $X\times Y$. The product inequality
for mean dimension is well known: $\operatorname{\mathrm{mdim}}(X\times
Y,G)\leq\operatorname{\mathrm{mdim}}(X,G)+\operatorname{\mathrm{mdim}}(Y,G)$,
while it was unknown for a long time if the product inequality could be an
equality. In 2019, Masaki Tsukamoto constructed the first example of two
different continuous actions of $G$ on compact metrizable spaces $X$ and $Y$,
respectively, such that the product inequality becomes strict. However, there
is still one longstanding problem which remains open in this direction, asking
if there exists a continuous action of $G$ on some compact metrizable space
$X$ such that $\operatorname{\mathrm{mdim}}(X\times
X,G)<2\cdot\operatorname{\mathrm{mdim}}(X,G)$. We solve this problem. Somewhat
surprisingly, we prove, in contrast to (topological) dimension theory, a
rather satisfactory theorem: If an infinite (countable discrete) amenable
group $G$ acts continuously on a compact metrizable space $X$, then we have
$\operatorname{\mathrm{mdim}}(X^{n},G)=n\cdot\operatorname{\mathrm{mdim}}(X,G)$,
for any positive integer $n$. Our product formula for mean dimension, together
with the example and inequality (stated previously), eventually allows mean
dimension of product actions to be fully understood.

###### Key words and phrases:

Mean dimension; Product space.

###### 2010 Mathematics Subject Classification:

37B99; 54F45.

##  1\. Main result

Mean dimension is a topological invariant of dynamical systems, which
originates with Mikhail Gromov in 1999 and which was investigated with deep
applications around 2000 by Elon Lindenstrauss and Benjamin Weiss within the
framework of amenable group actions. The purpose of this paper is to establish
a fundamental formula for mean dimension of product actions. We shall state
our main theorem very quickly in this section (Section 1). The definition of
mean dimension and all the necessary terminologies can be found in Section 2.
The proof of the main result is located in Section 3.

Let us start with convention. Throughout this paper the symbol $\mathbb{N}$
will denote the set of positive integers. All acting groups are always assumed
to be countable and discrete. If an amenable group $G$ acts continuously on a
compact metrizable space $X$ then we denote its mean dimension by
$\operatorname{\mathrm{mdim}}(X,G)$, which takes values in $[0,+\infty]$.

Let an amenable group $G$ act continuously on compact metrizable spaces
$X_{i}$, respectively, where $i$ ranges over some subset $I$ of $\mathbb{N}$.
Consider the product action of $G$ on the product space $\prod_{i\in I}X_{i}$.
The product inequality for mean dimension (due to Lindenstrauss and Weiss
[LW00]) is well known:

$\operatorname{\mathrm{mdim}}(\prod_{i\in I}X_{i},G)\leq\sum_{i\in
I}\operatorname{\mathrm{mdim}}(X_{i},G).$

Nevertheless, it was unknown for a long time if the product inequality could
always be an equality. In 2019, Masaki Tsukamoto [Tsu19] successfully
constructed the first example of two different continuous actions of $G$ on
compact metrizable spaces $X$ and $Y$, respectively, such that the product
inequality becomes strict:

$\operatorname{\mathrm{mdim}}(X\times
Y,G)<\operatorname{\mathrm{mdim}}(X,G)+\operatorname{\mathrm{mdim}}(Y,G).$

A serious reader may observe that in order to have a full understanding of
mean dimension of product actions, there is still one longstanding issue that
remains open, asking if it is possible for the two continuous actions $(X,G)$
and $(Y,G)$ (mentioned in the above example) to be essentially the same.
Formally, we study the problem in this direction as follows:

  * •

For an (arbitrarily fixed) amenable group $G$, does there exist a continuous
action of $G$ on some compact metrizable space $X$ such that

$\operatorname{\mathrm{mdim}}(X\times
X,G)<2\cdot\operatorname{\mathrm{mdim}}(X,G)?$

We solve this problem completely.

First of all, let us make two observations on this issue here. On the one
hand, we note that in dimension theory there is an example (to be precise, we
refer to Lemma 2.3) of a compact metrizable space $K$ of (topological)
dimension $\dim(K)$ so that the product space $K\times K$ satisfies

$\dim(K\times K)<2\cdot\dim(K).$

On the other hand, we have made mention of Tsukamoto’s example which is highly
similar to such a classically known analogue that takes place in dimension
theory.

Apparently, both of these two notable phenomena lead naturally to a seemingly
plausible impression, i.e. it would be true that we could finally find a
compact metrizable space $X$ (with a continuous action of $G$ on $X$)
satisfying that $\operatorname{\mathrm{mdim}}(X\times X,G)$ is strictly less
than $2\cdot\operatorname{\mathrm{mdim}}(X,G)$, as the (former) example in
dimension theory would stimulate us to strengthen the (latter) construction of
Tsukamoto’s example in mean dimension theory with the help of some
sufficiently refined method (which seems to be hopeful and which might be
technically difficult).

However, this assertion turns out to be false. Somewhat surprisingly, we prove
a rather satisfactory theorem:

######  Theorem 1.1 (Main theorem).

If an infinite amenable group $G$ acts continuously on a compact metrizable
space $X$, then we have

$\operatorname{\mathrm{mdim}}(X^{n},G)=n\cdot\operatorname{\mathrm{mdim}}(X,G)$

for all positive integers $n$.

######  Remark 1.2.

Theorem 1.1 also applies to $n\in\mathbb{N}\cup\\{0\\}\cup\\{+\infty\\}$
provided $0\cdot(+\infty)=(+\infty)\cdot 0=0$. Indeed, this statement is
obviously correct for $n=0$ if we set $X^{0}$ to be the one-point set.
Moreover, with a slightly more effort assuming Theorem 1.1 for any
$n\in\mathbb{N}$ we are able to show that the statement is true for
$n=+\infty$. In fact, there are two cases. If
$\operatorname{\mathrm{mdim}}(X,G)=0$, then
$\operatorname{\mathrm{mdim}}(X^{\infty},G)=0$ follows directly from the
product inequality for mean dimension. Now we suppose
$\operatorname{\mathrm{mdim}}(X,G)>0$. Since it is clear that
$\operatorname{\mathrm{mdim}}(X^{\infty},G)\geq\operatorname{\mathrm{mdim}}(X^{n},G)=n\cdot\operatorname{\mathrm{mdim}}(X,G)$
for every $n\in\mathbb{N}$ (by definition and by the statement of Theorem 1.1
for all $n\in\mathbb{N}$), we have
$\operatorname{\mathrm{mdim}}(X^{\infty},G)=+\infty$.

######  Remark 1.3.

If $G$ is a finite group (which is automatically amenable) then Theorem 1.1
may be false. Notice that in this case we have by definition
$\operatorname{\mathrm{mdim}}(X,G)=\dim(X)/|G|$. As follows is an entire
picture of the situation: If $X$ satisfies $\dim(X)=+\infty$, then
$\operatorname{\mathrm{mdim}}(X,G)=+\infty$. So does any of its self-product.
Thus, the statement remains true for every $n\in\mathbb{N}\cup\\{0,+\infty\\}$
in this case. Now let us suppose that $X$ is finite dimensional. It follows
from Lemma 2.3 that for each $n\in\mathbb{N}$

$\displaystyle\operatorname{\mathrm{mdim}}(X^{n},G)=\begin{cases}n\cdot\operatorname{\mathrm{mdim}}(X,G),&\text{if
$\dim(X\times X)=2\dim(X)$}\\\
n\cdot\operatorname{\mathrm{mdim}}(X,G)-(n-1)/|G|,&\text{otherwise}\end{cases}.$

Thus, in this case the statement fails if and only if $X$ does not satisfy
$\dim(X\times X)=2\dim(X)$ and meanwhile $n$ does not belong to
$\\{0,1,+\infty\\}$. In short, the exact range to which the statement of
Theorem 1.1 does not apply is where $G$ is a finite group, $X$ satisfies
$\dim(X\times X)<2\dim(X)$, and $n\in\mathbb{N}\setminus\\{1\\}$.

In contrast to dimension theory, Theorem 1.1 enables an unexpected behaviour
in mean dimension theory to become clarified. Furthermore, our main theorem,
together with Lindenstrauss–Weiss’ inequality and Tsukamoto’s example (stated
previously), eventually allows mean dimension of product actions to be fully
understood.

Our result is new even for $\mathbb{Z}$-actions. A novel point of the theorem
is that the statement applies to the context of amenable group actions,
whereas the proof goes through the framework of its sofic nature. The key
ingredient of our idea is to produce different sofic approximation sequences
for the acting group, with respect to which, we consider the sofic mean
dimension of a group action.

##  2\. A brief review of mean dimension

Both mean dimension and sofic groups originate with Misha Gromov around 1999.
A systematic study of mean dimension in the context of amenable group actions
was given around 2000 by Lindenstrauss and Weiss [LW00]. In 2013, Hanfeng Li
[Li13] introduced the notion of sofic mean dimension which is a successful
extension of the definition of mean dimension to the setting of sofic group
actions, and further, Li built its connection with classical mean dimension.
This section is devoted to all the precise notions and notations in relation
to our result, and to collecting fundamental material on them.

###  2.1. Sofic groups

We denote by $|F|$ the cardinality of a set $F$. For every $d\in\mathbb{N}$ we
write $[d]$ for the set $\\{k\in\mathbb{N}:1\leq k\leq d\\}$ and
$\operatorname{\mathrm{Sym}}(d)$ for the group of permutations of $[d]$. A
group $G$ is sofic if there is a sequence

$\Sigma=\\{\sigma_{i}:G\to\operatorname{\mathrm{Sym}}(d_{i})\\}_{i\in\mathbb{N}}$

together with a sequence $\\{d_{i}\\}_{i\in\mathbb{N}}\subset\mathbb{N}$ such
that the following three conditions are satisfied:

$\displaystyle\bullet\quad\quad\lim_{i\to\infty}\frac{1}{d_{i}}|\\{k\in[d_{i}]:\sigma_{i}(st)(k)=\sigma_{i}(s)\sigma_{i}(t)(k)\\}|=1\quad\text{for
all $s,t\in G$;}$
$\displaystyle\bullet\quad\quad\lim_{i\to\infty}\frac{1}{d_{i}}|\\{k\in[d_{i}]:\sigma_{i}(s)(k)\neq\sigma_{i}(t)(k)\\}|=1\quad\text{for
all distinct $s,t\in G$;}$
$\displaystyle\bullet\quad\quad\lim_{i\to\infty}d_{i}=+\infty.$

Such a sequence $\Sigma$ is called a sofic approximation sequence for $G$.

######  Remark 2.1.

Note that the third condition will be fulfilled automatically if $G$ is an
infinite group.

######  Remark 2.2.

The sofic groups are a fairly extensive class, which contain in particular all
amenable groups and all residually finite groups. However, it has not yet been
confirmed if there exists a non-sofic group.

###  2.2. Product actions

Let $G$ be a group. By the terminology “$G$ acts continuously on a compact
metrizable space $X$” we understand a continuous mapping

$\Phi:G\times X\to X,\quad(g,x)\mapsto gx$

satisfying

$\Phi(e,x)=x,\quad\Phi(gh,x)=\Phi(g,\Phi(h,x)),\quad\forall x\in X,\;\forall
g,h\in G,$

where $e$ is the identity element of the group $G$.

Let a group $G$ act continuously on compact metrizable spaces $X_{n}$,
respectively, where $n$ ranges over some
$R\in\\{[r]:r\in\mathbb{N}\\}\cup\\{\mathbb{N}\\}$. The product action of $G$
on the product space $\prod_{n\in R}X_{n}$ is defined as follows:

$g(x_{n})_{n\in R}=(gx_{n})_{n\in R},\quad\forall g\in
G,\;\forall(x_{n})_{n\in R}\in\prod_{n\in R}X_{n}.$

###  2.3. Dimension

We denote by $\dim(K)$ the topological dimension (i.e. the Lebesgue covering
dimension) of a compact metrizable space $K$. If the space $K$ is empty, then
we set $\dim(K)=-\infty$. For a finite dimensional (nonempty) compact
metrizable space $K$, since it was classically known that

$2\dim(K)-1\leq\dim(K\times K)\leq 2\dim(K)$

and since $\dim(K)$ must be a nonnegative integer, we have

  * •

either $\quad\dim(K\times K)=2\dim(K)$,

  * •

or $\quad\dim(K\times K)=2\dim(K)-1$.

For a friendly treatment of the following result in dimension theory we refer
to [Tsu19, Theorem 2.5].

######  Lemma 2.3.

Let $K$ be a finite dimensional compact metrizable space. Then for every
$n\in\mathbb{N}$

$\dim(K^{n})=\begin{cases}n\dim(K),&\text{if $K$ satisfies $\dim(K\times
K)=2\dim(K)$},\\\ n\dim(K)-n+1,&\text{otherwise}.\end{cases}$

Let $X$ and $P$ be two compact metrizable spaces. Let $\rho$ be a compatible
metric on $X$. For $\epsilon>0$ a continuous mapping $f:X\to P$ is called an
$\epsilon$-embedding with respect to $\rho$ if $f(x)=f(x^{\prime})$ implies
$\rho(x,x^{\prime})<\epsilon$, for all $x,x^{\prime}\in X$. Let
$\operatorname{\mathrm{Widim}}_{\epsilon}(X,\rho)$ be the minimum topological
dimension $\dim(P)$ of a compact metrizable space $P$ which admits an
$\epsilon$-embedding $f:X\to P$ with respect to $\rho$.

######  Remark 2.4.

We may verify that the topological dimension of $X$ may be recovered by
$\dim(X)=\lim_{\epsilon\to
0}\operatorname{\mathrm{Widim}}_{\epsilon}(X,\rho)$.

Let $K$ be a compact metrizable space with a compatible metric $\rho$. For
every $n\in\mathbb{N}$ we define on the product space $K^{n}$ two compatible
metrics $\rho_{2}$ and $\rho_{\infty}$ as follows:

$\rho_{2}\left((x_{i})_{i\in[n]},(y_{i})_{i\in[n]}\right)=\sqrt{\frac{1}{n}\sum_{i\in[n]}(\rho(x_{i},y_{i}))^{2}},$
$\rho_{\infty}\left((x_{i})_{i\in[n]},(y_{i})_{i\in[n]}\right)=\max_{i\in[n]}\rho(x_{i},y_{i}).$

We do not include $n\in\mathbb{N}$ in the notations $\rho_{2}$ and
$\rho_{\infty}$ because it does not cause any ambiguity.

###  2.4. Mean dimension

A group $G$ is amenable if there exists a sequence
$\\{F_{n}\\}_{n\in\mathbb{N}}$ of nonempty finite subsets of $G$ such that for
any $g\in G$

$\lim_{n\to\infty}\frac{|F_{n}\triangle gF_{n}|}{|F_{n}|}=0.$

Such a sequence $\\{F_{n}\\}_{n\in\mathbb{N}}$ is called a Følner sequence of
the group $G$.

Let an amenable group $G$ act continuously on a compact metrizable space $X$.
Take a Følner sequence $\\{F_{n}\\}_{n\in\mathbb{N}}$ of $G$ and a compatible
metric $\rho$ on $X$. For a nonempty finite subset $F$ of $G$ we set

$\rho_{F}(x,x^{\prime})=\rho_{\infty}\left((gx)_{g\in F},(gx^{\prime})_{g\in
F}\right),\quad\forall\,x,x^{\prime}\in X.$

It is clear that $\rho_{F}$ is also a compatible metric on $X$. The mean
dimension of $(X,G)$ is defined by

$\operatorname{\mathrm{mdim}}(X,G)=\lim_{\epsilon\to
0}\lim_{n\to\infty}\frac{\operatorname{\mathrm{Widim}}_{\epsilon}(X,\rho_{F_{n}})}{|F_{n}|}.$

It is well known that the limits in the above definition always exist. The
value $\operatorname{\mathrm{mdim}}(X,G)$ is independent of the choices of a
Følner sequence $\\{F_{n}\\}_{n\in\mathbb{N}}$ of $G$ and a compatible metric
$\rho$ on $X$.

###  2.5. Sofic mean dimension

Suppose that
$\Sigma=\\{\sigma_{i}:G\to\operatorname{\mathrm{Sym}}(d_{i})\\}_{i\in\mathbb{N}}$
is a sofic approximation sequence for a sofic group $G$ which acts
continuously on a compact metrizable space $X$ equipped with a compatible
metric $\rho$. For a finite subset $F$ of $G$, $\delta>0$ and a map
$\sigma:G\to\operatorname{\mathrm{Sym}}(d)$ (where $d\in\mathbb{N}$) we define

$\operatorname{\mathrm{Map}}(\rho,F,\delta,\sigma)=\\{\phi:[d]\to
X:\rho_{2}(\phi\circ\sigma(s),s\phi)\leq\delta,\,\forall s\in F\\}.$

We consider the set $\operatorname{\mathrm{Map}}(\rho,F,\delta,\sigma)$ as a
compact subspace of the product space $X^{d}$. The sofic mean dimension of
$(X,G)$ with respect to $\Sigma$ is defined by

$\operatorname{\mathrm{mdim}}_{\Sigma}(X,G)=\sup_{\epsilon>0}\inf_{F\subset
G\text{ finite,
}\,\delta>0}\limsup_{i\to\infty}\frac{\operatorname{\mathrm{Widim}}_{\epsilon}\left(\operatorname{\mathrm{Map}}(\rho,F,\delta,\sigma_{i}),\rho_{\infty}\right)}{d_{i}}.$

The definition of $\operatorname{\mathrm{mdim}}_{\Sigma}(X,G)$ does not depend
on the compatible metrics $\rho$ on $X$. Nevertheless, it is not clear yet if
there is an example of a sofic approximation sequence $\Sigma^{\prime}$
different from $\Sigma$, which leads to a different value
$\operatorname{\mathrm{mdim}}_{\Sigma^{\prime}}(X,G)$. We shall make use of
the following theorem [Li13, Section 3].

######  Lemma 2.5.

If an infinite amenable group $G$ acts continuously on a compact metrizable
space $X$ and if $\Sigma$ is a sofic approximation sequence for $G$, then
$\operatorname{\mathrm{mdim}}_{\Sigma}(X,G)=\operatorname{\mathrm{mdim}}(X,G)$.

##  3\. Proof of the main theorem

Let $G$ be an infinite amenable group which acts continuously on a compact
metrizable space $X$. We fix a positive integer $n$ in this section. Recall
that $(X^{n},G)$ denotes the product action of $G$ on the product space
$X^{n}$. We shall prove

$\operatorname{\mathrm{mdim}}(X^{n},G)=n\cdot\operatorname{\mathrm{mdim}}(X,G).$

Since the group $G$ is amenable, it is sofic. Therefore we may take a sofic
approximation sequence for $G$:

$\Sigma=\\{\sigma_{i}:G\to\operatorname{\mathrm{Sym}}(d_{i})\\}_{i\in\mathbb{N}},$

where $\\{d_{i}\\}_{i\in\mathbb{N}}$ is a sequence of positive integers with
$d_{i}\to+\infty$ as $i\to+\infty$. We generate a new sofic approximation
sequence for $G$ (confirmed below) as follows:

$\Sigma^{(n)}=\\{\sigma^{(n)}_{i}:G\to\operatorname{\mathrm{Sym}}(nd_{i})\\}_{i\in\mathbb{N}},$

where for every $i\in\mathbb{N}$ the map

$\sigma^{(n)}_{i}:G\to\operatorname{\mathrm{Sym}}(nd_{i})$

is defined by:

$\sigma^{(n)}_{i}(g)\left((j-1)n+l\right)=(\sigma_{i}(g)(j)-1)n+l,\,\quad\;\forall
g\in G,\;\forall j\in[d_{i}],\;\forall l\in[n].$

######  Lemma 3.1.

$\Sigma^{(n)}$ is a sofic approximation sequence for $G$.

###### Proof.

Clearly, for every $i\in\mathbb{N}$ and $g\in G$ the map
$\sigma^{(n)}_{i}(g):[nd_{i}]\to[nd_{i}]$ is a permutation of $[nd_{i}]$.
Besides, it is straightforward to verify that for any $s,t\in G$,
$j\in[d_{i}]$ and $l\in[n]$ we have

$\sigma^{(n)}_{i}(st)\left((j-1)n+l\right)=\sigma^{(n)}_{i}(s)\sigma^{(n)}_{i}(t)\left((j-1)n+l\right)\iff\sigma_{i}(st)(j)=\sigma_{i}(s)\sigma_{i}(t)(j),$
$\sigma^{(n)}_{i}(s)\left((j-1)n+l\right)=\sigma^{(n)}_{i}(t)\left((j-1)n+l\right)\iff\sigma_{i}(s)(j)=\sigma_{i}(t)(j).$

Since $\Sigma$ is a sofic approximation sequence for $G$, the assertion
follows. ∎

Let us consider the sofic mean dimension of $(X,G)$ and $(X^{n},G)$ with
respect to the sofic approximation sequences $\Sigma^{(n)}$ and $\Sigma$,
respectively. These two values share the following relation.

######  Lemma 3.2 (Key lemma).

$\operatorname{\mathrm{mdim}}_{\Sigma^{(n)}}(X,G)=\frac{1}{n}\cdot\operatorname{\mathrm{mdim}}_{\Sigma}(X^{n},G).$

###### Proof.

We fix a compatible metric $\rho$ on $X$ in the proof. Let $\rho^{(n)}$ be the
compatible metric $\rho_{\infty}$ on $X^{n}$. Let us consider two compact
metric spaces as follows: $(X,\rho)$ and $(X^{n},\rho^{(n)})$. We take
$\epsilon>0$, $\delta>0$, a finite subset $F$ of $G$, and a positive integer
$i$, arbitrarily and fix them temporarily.

We note that both of the following two sets:

$\operatorname{\mathrm{Map}}(\rho,F,\delta,\sigma^{(n)}_{i})=\\{\phi:[nd_{i}]\to
X:\rho_{2}(\phi\circ\sigma^{(n)}_{i}(s),s\phi)\leq\delta,\,\forall s\in F\\}$
$\operatorname{\mathrm{Map}}(\rho^{(n)},F,\delta,\sigma_{i})=\\{\phi:[d_{i}]\to
X^{n}:\rho^{(n)}_{2}(\phi\circ\sigma_{i}(s),s\phi)\leq\delta,\,\forall s\in
F\\}$

can be regarded as compact subspaces of the product space
$X^{nd_{i}}=(X^{n})^{d_{i}}$. More explicitly, the point here is that we
identify $X^{nd_{i}}$ with $(X^{n})^{d_{i}}$. We notice that the construction
of the sofic approximation sequence $\Sigma^{(n)}$ for $G$ and the definition
of the product action $(X^{n},G)$ ensure that the terms
$\phi\circ\sigma^{(n)}_{i}(s)$ and $\phi\circ\sigma_{i}(s)$ agree, i.e.

$\phi\circ\sigma^{(n)}_{i}(s)=\phi\circ\sigma_{i}(s),\quad\;\forall\phi\in
X^{nd_{i}}=(X^{n})^{d_{i}},\;\forall s\in F.$

Further, we also remark that $\rho_{\infty}$ defined on $X^{nd_{i}}$
corresponds to $\rho^{(n)}_{\infty}$ defined on $(X^{n})^{d_{i}}$, namely

$\rho_{\infty}(\psi,\psi^{\prime})=\rho^{(n)}_{\infty}(\psi,\psi^{\prime}),\quad\,\forall\psi,\psi^{\prime}\in
X^{nd_{i}}=(X^{n})^{d_{i}},$

while $\rho_{2}$ defined on $X^{nd_{i}}$ and $\rho^{(n)}_{2}$ defined on
$(X^{n})^{d_{i}}$ satisfy the inequality:

$\frac{1}{\sqrt{n}}\cdot\rho^{(n)}_{2}(\psi,\psi^{\prime})\leq\rho_{2}(\psi,\psi^{\prime})\leq\rho^{(n)}_{2}(\psi,\psi^{\prime}),\quad\,\forall\psi,\psi^{\prime}\in
X^{nd_{i}}=(X^{n})^{d_{i}}.$

The above observation implies that

$\operatorname{\mathrm{Map}}(\rho^{(n)},F,\delta,\sigma_{i})\subset\operatorname{\mathrm{Map}}(\rho,F,\delta,\sigma^{(n)}_{i})\subset\operatorname{\mathrm{Map}}(\rho^{(n)},F,\sqrt{n}\delta,\sigma_{i}).$

It follows that

$\displaystyle\operatorname{\mathrm{Widim}}_{\epsilon}\left(\operatorname{\mathrm{Map}}(\rho^{(n)},F,\delta,\sigma_{i}),\rho^{(n)}_{\infty}\right)$
$\displaystyle\leq$
$\displaystyle\operatorname{\mathrm{Widim}}_{\epsilon}\left(\operatorname{\mathrm{Map}}(\rho,F,\delta,\sigma^{(n)}_{i}),\rho_{\infty}\right)$
$\displaystyle\leq$
$\displaystyle\operatorname{\mathrm{Widim}}_{\epsilon}\left(\operatorname{\mathrm{Map}}(\rho^{(n)},F,\sqrt{n}\delta,\sigma_{i}),\rho^{(n)}_{\infty}\right).$

Since $\epsilon>0$, $\delta>0$, a finite subset $F\subset G$ and
$i\in\mathbb{N}$ (which we took in the beginning of the proof) are arbitrary,
we deduce that

$\operatorname{\mathrm{mdim}}_{\Sigma^{(n)}}(X,G)=\frac{1}{n}\cdot\operatorname{\mathrm{mdim}}_{\Sigma}(X^{n},G).$

Thus, we end the proof. ∎

######  Remark 3.3.

The equality established in Lemma 3.2 is generally true for all sofic group
actions $(X,G)$ and all positive integers $n$. The acting group $G$ in this
lemma is not required to be infinite.

We are now able to prove Theorem 1.1. The key lemma (Lemma 3.2) indicates that

$\operatorname{\mathrm{mdim}}_{\Sigma^{(n)}}(X,G)=\frac{1}{n}\cdot\operatorname{\mathrm{mdim}}_{\Sigma}(X^{n},G).$

By Lemma 2.5, sofic mean dimension (with respect to any sofic approximation
sequence) will coincide with (classical) mean dimension, as the acting group
$G$ is infinite. Thus, we conclude with

$\operatorname{\mathrm{mdim}}(X,G)=\frac{1}{n}\cdot\operatorname{\mathrm{mdim}}(X^{n},G).$

######  Remark 3.4.

We explain shortly about the difficulty with this problem. Let $G$ be an
infinite amenable group which acts continuously on a compact metrizable space
$X$. We fix a positive integer $n$ and a Følner sequence
$\\{F_{k}\\}_{k\in\mathbb{N}}$ of $G$. We recall that

$\operatorname{\mathrm{mdim}}(X^{n},G)=\lim_{\epsilon\to
0}\lim_{n\to\infty}\frac{\operatorname{\mathrm{Widim}}_{\epsilon}(X^{n},(\rho_{\infty})_{F_{k}})}{|F_{k}|},$
$n\cdot\operatorname{\mathrm{mdim}}(X,G)=\lim_{\epsilon\to
0}\lim_{n\to\infty}\frac{n\cdot\operatorname{\mathrm{Widim}}_{\epsilon}(X,\rho_{F_{k}})}{|F_{k}|},$

where $\rho$ and $\rho_{\infty}$ are compatible metrics on $X$ and $X^{n}$,
respectively. To show

$\operatorname{\mathrm{mdim}}(X^{n},G)\geq
n\cdot\operatorname{\mathrm{mdim}}(X,G),$

a main issue is how to estimate the term
$n\cdot\operatorname{\mathrm{Widim}}_{\epsilon}(X,\rho_{F_{k}})$ from above
with terms such as some variants of
$\operatorname{\mathrm{Widim}}_{\epsilon}(X^{n},(\rho_{\infty})_{F_{k}})$. We
overcome this obstacle. The strategy we adopted is to consider different
approximation sequences in the limits. For a systematic treatment we went
through an approach of sofic mean dimension. To make it clearer, let us focus
on the case of $\mathbb{Z}$-actions. More precisely, let $(X,d)$ be a compact
metric space and $T:X\to X$ a homeomorphism. For convenience we change our
notations here, which apply only to the remark. For every positive integer $k$
we write $d_{k}$ for the compatible metric on $X$ defined by

$d_{k}(x,x^{\prime})=\max_{0\leq
i<k}d(T^{i}x,T^{i}x^{\prime}),\quad\forall\,x,x^{\prime}\in X.$

For simplicity we assume $n=2$. The product action of $\mathbb{Z}$ on the
product space $X\times X$ is denoted by $T\times T$. We take a compatible
metric $d\times d$ on $X\times X$ as follows:

$d\times
d\,((x_{1},x_{2}),(x^{\prime}_{1},x^{\prime}_{2}))=\max\\{d(x_{1},x^{\prime}_{1}),d(x_{2},x^{\prime}_{2})\\},\,\quad\forall\,(x_{1},x_{2}),\,(x^{\prime}_{1},x^{\prime}_{2})\;\in
X\times X.$

To estimate $2\cdot\operatorname{\mathrm{mdim}}(X,T)$ from above, we have to
turn to an alternative expression (replacing $N$ by $2N$ in the midst of the
equality below):

$2\cdot\operatorname{\mathrm{mdim}}(X,T)=\lim_{\epsilon\to
0}\lim_{N\to+\infty}\frac{2\cdot\operatorname{\mathrm{Widim}}_{\epsilon}(X,d_{N})}{N}=\lim_{\epsilon\to
0}\lim_{N\to+\infty}\frac{\operatorname{\mathrm{Widim}}_{\epsilon}(X,d_{2N})}{N}.$

Therefore, in order to show

$2\cdot\operatorname{\mathrm{mdim}}(X,T)\leq\operatorname{\mathrm{mdim}}(X\times
X,T\times T)=\lim_{\epsilon\to
0}\lim_{N\to+\infty}\frac{\operatorname{\mathrm{Widim}}_{\epsilon}(X\times
X,(d\times d)_{N})}{N}$

it suffices to prove

$\operatorname{\mathrm{Widim}}_{\epsilon}(X,d_{2N})\leq\operatorname{\mathrm{Widim}}_{\epsilon}(X\times
X,(d\times d)_{N})$

for $\epsilon>0$ and $N\in\mathbb{N}$. This will be deduced from the following
statement: The continuous mapping

$X\to X\times X,\quad x\mapsto(x,T^{N}x)$

is distance-increasing (actually it is distance-preserving) with respect to
$(X,d_{2N})$ and $(X\times X,(d\times d)_{N})$, i.e.

$d_{2N}(x,x^{\prime})=\max_{0\leq i\leq
2N-1}d(T^{i}x,T^{i}x^{\prime})=(d\times
d)_{N}\left((x,T^{N}x),(x^{\prime},T^{N}x^{\prime})\right),\quad\forall\,x,x^{\prime}\in
X.$

## References

  * [Li13] Hanfeng Li. Sofic mean dimension. Advances in Mathematics 244 (2013), 570–604.
  * [LW00] Elon Lindenstrauss, Benjamin Weiss. Mean topological dimension. Israel Journal of Mathematics 115 (2000), 1–24.
  * [Tsu19] Masaki Tsukamoto. Mean dimension of full shifts. Israel Journal of Mathematics 230 (2019), 183–193.