# Towards Zero Memory Footprint Spiking Neural Network Training

Bin Lei1 Sheng Lin Pei-Hung Lin2 Chunhua Liao2 Caiwen Ding1
1University of Connecticut 2Lawrence Livermore National Laboratory
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Biologically-inspired Spiking Neural Networks (SNNs), processing information
using discrete-time events known as spikes rather than continuous values, have
garnered significant attention due to their hardware-friendly and energy-
efficient characteristics. However, the training of SNNs necessitates a
considerably large memory footprint, given the additional storage requirements
for spikes or events, leading to a complex structure and dynamic setup. In
this paper, to address memory constraint in SNN training, we introduce an
innovative framework, characterized by a remarkably low memory footprint. We
(i) design a reversible SNN node that retains a high level of accuracy. Our
design is able to achieve a $\mathbf{58.65\times}$ reduction in memory usage
compared to the current SNN node. We (ii) propose a unique algorithm to
streamline the backpropagation process of our reversible SNN node. This
significantly trims the backward Floating Point Operations Per Second (FLOPs),
thereby accelerating the training process in comparison to current reversible
layer backpropagation method. By using our algorithm, the training time is
able to be curtailed by $\mathbf{23.8\%}$ relative to existing reversible
layer architectures.

##  1 Introduction

As a bio-inspired neuromorphic computing representative, Spiking Neural
Network (SNN) has attracted considerable attention, in contrast to the high
computational complexity and energy consumption of traditional Deep Neural
Networks (DNNs) [28, 2, 25, 10]. SNN processes information using discrete-time
events known as spikes rather than continuous values, offering extremely
hardware-friendly and energy-efficient characteristics. For instance, in a
robot navigation task using Intel’s Loihi [1], SNN could achieve a 276$\times$
reduction in energy compared to a conventional DNN approach. Work [24] shows
that DNN consumes 111mJ and 1035mJ per sample on MNIST and CIFAR-10,
respectively, while SNN consumes only 0.66mJ and 102mJ, i.e., 168$\times$ and
10$\times$ energy reduction.

Despite their numerous advantages, one major bottleneck in the deployment of
SNNs has been memory consumption. The memory complexity of a traditional DNN
with a depth of $L$ is $\mathcal{O}(L)$. But for SNN of the same depth $L$,
there are several timesteps $T$ involved in the computation. Consequently, the
memory complexity of SNNs escalates to $\mathcal{O}(L*T)$. For instance, the
memory requirement during the DNN training process of ResNet19 is 0.6 GB, but
for the SNN with the same architecture could reach about 12.34 GB (~20
$\times$) when time-step equals 10. This presents a significant challenge for
their applicability to resource-constrained systems, such as IoT-Edge devices
[21].

To address the issue of high memory usage of SNN, researchers have proposed
several methods, including Quantization [21], Weight Sparsification [22, 12],
the Lottery Ticket Hypothesis [13], Knowledge Distillation [9], Efficient
Checkpointing [27], and so on. In this paper, we introduce a novel reversible
SNN node that drastically compresses the memory footprint of the SNN node
inside the entire network. Our method achieves state-of-the-art results in
terms of SNN memory savings. It achieves this by recalculating all the
intermediate states on-the-fly, rather than storing them during the backward
propagation process. To further enhance the efficiency of our approach, we
also present a new algorithm for the backpropagation process of our reversible
SNN node, which significantly reduces the training time compared with the
original reversible layer backpropagation method. Remarkably, our method
maintains the same level of accuracy throughout the process.

As a result of our innovations, we reduce the memory complexity of the SNN
node from $\mathcal{O}(n^{2})$ to $\mathcal{O}(1)$, while maintaining
comparable accuracy to traditional SNN node. Moreover, our method reduces the
FLOPs needed for the backpropagation by a factor of 23% compared to existing
reversible layer backpropagation method, thus accelerating the training
process. Collectively, these advances pave the way for more efficient and
scalable SNN implementations, enabling the deployment of these biologically
inspired networks across a wider range of applications and hardware platforms.

##  2 Background And Related Works

###  2.1 Spiking Neural Network

Spiking neural network (SNN) uses sparse binary spikes over multiple time
steps to deal with visual input in an event-driven manner [1, 3]. We use SNN
with the popular Leaky Integrate and Fire (LIF) spiking neuron. The forward
pass is formulated as follows.

$\displaystyle v[t]=\alpha v[t{-}1]+\sum_{i}w_{i}s_{i}[t]-\vartheta o[t-1]$
(1a) $\displaystyle o[t]=h(v[t]-\vartheta)$ (1b) $\displaystyle
h(x)=\begin{cases}0,&\text{if }x<0\\\ 1,&\text{otherwise}\end{cases}$ (1c)

where $t$ denotes time step. In Eq. (1a), $v[t]$ is the dynamics of the
neuron’s membrane potential after the trigger of a spike at time step t. The
sequence $s_{i}[t]\in\\{0,1\\}$ represents the $i$-th input spike train,
consisting solely of 0s and 1s, while $w_{i}$ is the corresponding weight. In
Eq. (1b), $o[t]\in\\{0,1\\}$ is the neuron’s output spike train. In Eq. (1c),
$h(x)$ is the Heaviside step function to generate the outputs.

In the backward pass, we adopt Backpropagation Through Time (BPTT) to train
SNNs. The BPTT for SNNs using a surrogate gradient [19], which is formulated
as follows.

$\displaystyle\delta_{l}[t]=\epsilon_{l+1}[t]w_{l+1}$ (2a)
$\displaystyle\epsilon_{l}[t]=\delta_{l}[t]\phi_{l}[t]+\alpha\epsilon_{l}[t]$
(2b) $\displaystyle\frac{\partial L}{\partial
w_{l}}=\sum_{t=0}^{T-1}\epsilon_{l}[t]\cdot[s_{l}[t]]^{\intercal}$ (2c)

Denote $L$ as the loss, in Eq. (2a), $\delta_{l}[t]{=}\frac{\partial
L}{\partial o_{l}[t]}$ is the error signal at layer $l$ time step $t$ which is
propagated using above formulations. In Eq. (2b),
$\epsilon_{l}[t]{=}\frac{\partial L}{\partial v_{l}[t]}$,
$\phi_{l}[t]{=}\frac{\partial o_{l}[t]}{\partial v_{l}[t]}{=}\frac{\partial
u(v_{l}[t]-\vartheta)}{\partial v_{l}[t]}$, where we follow the gradient
surrogate function in [7] to approximate the derivative of $u(x)$, such that
$\frac{\partial u(x)}{\partial x}\approx\frac{1}{1+\pi^{2}x^{2}}$. Eq. (2c)
calculates the gradient of $l^{th}$ layer weight $w_{l}$.

###  2.2 Reversible Layer

A reversible layer refers to a family of neural network architectures that are
based on the Non-linear Independent Components Estimation [4, 5]. In
traditional training methods, the activation values of each layer are stored
in memory, which leads to a dramatic increase in memory requirements as the
network depth increases. However, reversible transformation technology allows
us to store only the final output of the network and to recompute the
discarded ones when needed. This approach significantly reduces memory
requirements, making it possible to train deeper neural networks and more
complex models under limited memory conditions, thus potentially unlocking new
insights and improving performance across a wide range of tasks. The
reversible transformation has been used in different kinds of neural networks,
such as CNN [8], Graph neural networks (GNN) [15], Recurrent Neural Networks
(RNN) [17] and some transformers [18]. Furthermore, studies have demonstrated
the effectiveness of reversible layers in different types of tasks, including
image segmentation [20], natural language processing [14], compression [16],
and denoising [11].

##  3 Reversible SNN Algorithm

###  3.1 Reversible SNN Memory Analysis

During the training process of SNN networks, the activation values occupy the
main memory storage space. The SNN activation value memory analysis schematic
diagram is shown in Fig.1.

Figure 1: Activation value Memory Comparison between the original SNN network
and our reversible SNN network, using VGG13 with a timestep of ten as an
example.  : SNN node, : Save to memory, with ✗: NOT save to memory.

In this figure, we use the VGG-13 architecture [26] with ten timesteps as an
example. The percentage values represent the memory footprint ratio of each
part in the entire network. The left diagram is the original SNN where the
activation values of $X$ account for 90.9% of the memory usage, and the output
potentials of each neuron occupy 9.1% of the memory. The right diagram is our
designed reversible SNN, which only requires saving the final $X_{output}$
values and the output potentials of each neuron, without storing all
intermediate values, thus significantly saving memory. The intermediate
activation values will be regained during the backpropagation process through
our inverse calculation equation. In this example, our method is able to save
90.21% of the memory used for activation values. The exact amount of memory
saved by our method will be shown in Experiment part.

###  3.2 Reversible SNN Forward Calculation

Our forward algorithm is in the upper section of Fig.2.

Figure 2: This reversibility demo use $2\times 2$ toy Input as an example and
shows our forward and inverse calculations.  : The origin of the equations in
the inverse process.

❶: The various input states $S=(X,V)$ of each neuron are evenly divided into
two groups along the last dimension. Namely: $S=[S_{1},S_{2}]$.

❷: Calculate the first part of output $Y_{1}$:

$M_{1}^{t}=V_{1}^{t-1}+\frac{1}{\tau}\cdot\left(X_{1}^{t}-V_{1}^{t-1}\right)$
(3)

$Y_{1}^{t}=H\left(M_{1}^{t}-V_{th}\right)+\beta\cdot X_{2}^{t}$ (4)

$M_{1}^{t}$ is the membrane potential of the first half neuron at time $t$.
$V_{2}^{t-1}$ is the input potential of the second half neuron at time $t-1$.
$\tau$ is the time constant. $X_{2}^{t}$ is the input to the second half
neuron at time $t$. $V_{th}$ is the threshold voltage of the neurons. $H()$ is
the Heaviside step function. $\beta$ is a scaling factor for the input.
$\beta\cdot X_{2}^{t}$ will help $Y_{1}^{t}$ to collect information about the
second half of the input in the next step. Then calculate first part of output
voltage $V_{1}^{t}$:

$V_{1}^{t}=\left(1-Y_{1}^{t}\right)\odot M_{1}^{t}+Y_{1}^{t}\cdot
V_{res}+\alpha\cdot V_{1}^{t-1}$ (5)

$V_{1}^{t}$ is the output potential of the first half neuron at time $t$.
$V_{res}$ is the reset voltage of the neurons. $\alpha$ is a scaling factor
for the membrane potential.

❸: Use the first part of output $Y_{1}$ to calculate the second part $Y_{2}$:

$M_{2}^{t}=V_{2}^{t-1}+\frac{1}{\tau}\left(Y_{1}^{t}-V_{2}^{t-1}\right)$ (6)

$Y_{2}^{t}=H\left(M_{2}^{t}-V_{th}\right)+\beta\cdot X_{1}^{t}$ (7)

$M_{2}^{t}$ is the membrane potential of the second half neuron at time $t$.
$Y_{2}^{t}$ is the output of the second half neuron at time $t$. calculate the
second part of output voltage $V_{2}^{t}$:

$V_{2}^{t}=\left(1-Y_{2}^{t}\right)\odot M_{2}^{t}+Y_{2}^{t}\cdot
V_{res}+\alpha\cdot V_{2}^{t-1}$ (8)

$V_{2}^{t}$ is the output potential of the second half neuron at time $t$.

❹: For all the output states
$S_{output}=([Y_{1},Y_{2}],[V_{1}^{t},V_{2}^{t}])$, combine them by the last
dimension.

###  3.3 Reversible SNN Inverse Calculation

The purpose of the inverse calculation is to use the output results to obtain
the unsaved input values. i.e. Use $Y$ and $V_{output}$ to calculate $X$ and
$V$. Our inverse algorithm is in the lower section of Fig.2.

➀: For all the output states $S_{output}=(Y,V_{output})$, divide them into two
groups by the last dimension in the same way as in the first step of forward
calculation, namely: $S_{output}=[S_{output}1;S_{output}2]$

➁: Calculate $V_{1}^{t}$ by combine Eq.6 and Eq.8, simplify:

$V_{2}^{t-1}=\frac{V_{2}^{t}-(1-Y_{2})\cdot\frac{1}{\tau}\odot
Y_{1}-Y_{2}\cdot V_{reset}}{(1-Y_{2})\cdot(1-\frac{1}{\tau})+\alpha}$ (9)

Calculate $X_{1}^{t}$ by combine Eq.6 and 7, simplify:

$X_{1}^{t}=\left(Y_{2}^{t}-H\left(M_{2}^{t}-V_{th}\right)\right)\div\beta$
(10)

➂: Calculate $V_{1}^{t}$ by combine Eq.3 and Eq.5, simplify:

$V_{1}^{t-1}=\frac{V_{1}^{t}-(1-Y_{1})\cdot\frac{1}{\tau}\odot
X_{1}^{t}-Y_{1}\cdot V_{reset}}{(1-Y_{1})\cdot(1-\frac{1}{\tau})+\alpha}$ (11)

Calculate $X_{1}^{t}$ by combine Eq.3 and 4, simplify:

$X_{2}^{t}=\left(Y_{1}^{t}-H\left(M_{1}^{t}-V_{th}\right)\right)\div\beta$
(12)

➃: For all the input states $S=([X_{1},X_{2}],[V_{1}^{t-1},V_{2}^{t-1}])$,
combine them by the last dimension.

##  4 Inverse Gradient Calculation

Although our reversible architecture significantly reduces memory usage, it
does extend computation time for two primary reasons: (i) It necessitates the
recalculation of the activation values that weren’t originally stored. (ii)
Many of the previous reversible layer architectures have inherited the
backpropagation method from checkpointing [30, 6]. This method requires using
the recalculated intermediate activation values to rerun the forward equation,
thereby constructing a forward computational graph. This graph is then used to
derive the corresponding gradients. This step of rerunning the forward
equation introduces additional computational overhead, which extends the
overall computation time.

This scenario is prevalent across all existing architectures of reversible
layers, including Reversible GNN [15], Reversible RNN [17], Reversible
Transformers [18], and so on. To reduce the training time, we have designed a
new algorithm, called the inverse gradient calculation method, which is able
to substantially decrease the number of FLOPs during the backpropagation
process compared to the original reversible architecture. Our design is shown
in Fig.3.

Figure 3: Three different architectures for comparison. : Forward function, :
inverse function, : $\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X^{n-1}}}$
derivative, :Part of $\frac{\partial\mathbf{X^{n-1}}}{\partial\mathbf{X^{n}}}$
derivative, Brown variables: Cached values.

The left diagram illustrates the original forward and backward processes. The
middle diagram depicts the original calculation process for reversible layers.
It contains four steps:

  1. 1.

The input $X$ pass the forward function to compute the output $Y$, without
storing the input data to conserve memory.

  2. 2.

For each layer $n$: The output $X^{n}$ of this layer pass the inverse function
to compute the input $X^{n-1}$ of this layer. This process starts with the
final output $Y$.

  3. 3.

For each layer $n$: The input $X^{n-1}$ passes through the forward function
again to reconstruct the forward computational graph, which facilitates
gradient computation.

  4. 4.

For each layer $n$: Compute the gradient
$\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X^{n-1}}}$ based on the forward
computational graph.

The right diagram is our design, it contains three steps:

  1. 1.

The input $X$ pass the forward function to compute the output $Y$, without
storing the input data to conserve memory.

  2. 2.

For each layer $n$: The output $X^{n}$ of this layer pass the inverse function
to compute the input $X^{n-1}$ of this layer and construct an inverse
computational graph.

  3. 3.

For each layer $n$: Compute the gradient
$\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X^{n-1}}}$ based on the inverse
computational graph.

Below is the specific calculation formula of the
$\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X^{n-1}}}$ based on the inverse
computation graph, and the derivation process is in the Appendix.

$\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X_{1}^{n-1}}}=\frac{\theta}{2+\left(\pi\cdot\theta\cdot\left(M_{1}^{t}-V_{th}\right)\right)^{2}}\cdot\frac{1}{\tau}\odot\left(1+\frac{\theta}{2+\left(\pi\cdot\theta\cdot\left(M_{2}^{t}-V_{th}\right)\right)^{2}}\cdot\frac{1}{\tau}\right)+\beta$
(13)

$\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X_{2}^{n-1}}}=\frac{\theta}{2+\left(\pi\cdot\theta\cdot\left(M_{2}^{t}-V_{th}\right)\right)^{2}}+\beta$
(14)

All the variables in Eq.13 and Eq.14 have the same meaning as the variables in
Eq.3-Eq.12 and $\theta$ is an adjustable constant parameter.

The ability to perform computational graph inverse computation in my algorithm
is based on that our forward function has symmetry with the inverse
computation function.

For the original reversible network:

$FLOPS_{backward}^{ori}=FLOPS_{inverse}+FLOPS_{forward}+FLOPS_{\frac{\partial\mathbf{X^{n}}}{\partial\mathbf{X^{n-1}}}}$
(15)

For our reversible network:

$FLOPS_{backward}^{our}=FLOPS_{inverse}+FLOPS_{part~{}of\frac{\partial\mathbf{X^{n-1}}}{\partial\mathbf{X^{n}}}}$
(16)

Compared to the standard reversible network, our method reduces FLOPS by 23%.
The FLOPS analysis is shown in Appendix and the detailed time mearsurement is
shown in the Experiment part.

##  5 Experiment

We first compare our design with the SOTA SNN Memory-efficient methods for SNN
training on the several datasets. Subsequently, we incorporate our reversible
SNN node into different architectures across various datasets. The goal is
twofold: Firstly, we wanted to demonstrate that compared to the SNN node
currently in use, our reversible version is able to offer substantial memory
savings. Secondly, we aim to show that, when compared to the existing
reversible layer backpropagation method, our reversible SNN node
backpropagation design is able to considerably reduce the time spent in the
backpropagation process, thereby accelerating the training phase. In the final
part, we conduct an ablation study to evaluate the influence of various
parameters within our equations and the impact of the number of groups into
which the input is divided on the performance of our model.

All experiments were conducted on a Quadro RTX6000 GPU equipped with 24GB of
memory, using PyTorch 1.13.1 with CUDA 11.4, and an Intel(R) Xeon(R) Gold 6244
CPU running at 3.60GHz. To ensure that the values we obtain through inverse-
calculation is the same as the original forward-calculation method, we use
torch.allclose(rtol=$1e^{-06}$, atol=$1e^{-10}$) to compare all the inverse
calculated values with the origianl forward calculated values. All the results
return true as they should be. Detailed hyperparameter settings for each
experiment are provided in the Appendix.

###  5.1 Comparison with the SOTA Methods

We conducted a comparison of our approach with the current SOTA methods in
memory efficiency during the SNN training process on the CIFAR10 and CIFAR100
datasets. To verify the universality of our work, we apply our designed
reversible SNN node to the current SOTA sparse training work for SNNs. A
comparison was then made between these two methods on the Tiny-ImageNet
dataset, the results are shown in Table 1.

Table 1: Comparison of our work with the SOTA methods in memory efficiency
during the SNN training process. For all the works: Batch size = 128. *: They
did not provide the memory data directly for training CIFAR100, we estimate it
based on their memory usage for training CIFAR10 and their parameter data.

Dataset | Method | Architecture | Time-steps | Accuracy | Memory(GiB)
---|---|---|---|---|---
| OTTT [32] | VGG(sWS) | 6 | 93.52% | 4
CIFAR10 | S2A-STSU [29] | ResNet-17 | 5 | 92.75% | 27.93
IDE-LIF [33] | CIFARNet-F | 30 | 91.74% | 2.8
Hybrid [23] | VGG-16 | 100 | 91.13% | 9.36
Tandem [31] | CifarNet | 8 | 89.04% | 4.2
Skipper [27] | VGG-5 | 100 | 87.44% | 4.6
RevSNN(Ours) | ResNet-18 | 4 | 91.87% | 1.101
CIFAR100 | IDE-LIF [33] | CIFARNet-F | 30 | 71.56% | 3.51*
OTTT [32] | VGG(sWS) | 6 | 71.05% | 4.04*
S2A-STSU [29] | VGG-13 | 4 | 68.96% | 31.05
Skipper [27] | VGG-5 | 100 | 66.48% | 4.6
RevSNN(Ours) | ResNet-18 | 4 | 71.13% | 1.12
Tiny-ImageNet | ND(Dense) [12] | VGG-16 | 5 | 39.45% | 3.99
ND(90% Sparsity) [12] | VGG-16 | 5 | 39.12% | 3.78
ND(99% sparsity) [12] | VGG-16 | 5 | 33.84% | 3.76
RevND(Ours) | VGG-16 | 5 | 39.73% | 2.01
ND(Dense) [12] | ResNet-19 | 5 | 50.32% | 5.29
ND(90% Sparsity) [12] | ResNet-19 | 5 | 49.25% | 5.11
ND(99% sparsity) [12] | ResNet-19 | 5 | 41.96% | 5.09
RevND(Ours) | ResNet-19 | 5 | 50.63% | 2.47

Our approach (RevSNN) achieves a $\mathbf{2.54\times}$ memory reduction on the
CIFAR10 dataset and a $\mathbf{3.13\times}$ memory reduction on the CIFAR100
dataset compared to the current SOTA SNN training memory-efficient method. At
the same time, we also maintain a high level of accuracy. On the Tiny-ImageNet
dataset, we only replaced the original SNN node with our designed reversible
SNN node, keeping all other conditions consistent (RevND). As a result, the
accuracy of our VGG-16 model structure is 0.28% points higher than that of the
original dense model and saves $\mathbf{1.87\times}$ more memory than the
original work at 99% sparsity. On the ResNet-19 model, our accuracy is 0.31%
points higher than the dense model, and saves $\mathbf{2.06\times}$ more
memory than the original work at 99% sparsity.

###  5.2 Memory Consumption Evaluation

To investigate whether our newly designed reversible SNN node achieves the
expected memory savings compared to the original Spiking Neural Node, we
incorporated our node into a range of architectures including VGG-11, 13, 16,
19, and ResNet-19, 34, 50, 101. For the VGG architectures, we examined the
corresponding memory usage for timesteps ranging from 1 to 20, while for the
ResNet architectures, we scrutinized the memory usage for timesteps from 1 to
10. These tests were conducted on the CIFAR-10 dataset. For all the
experiments, we keep the batch size = 128.

The SNN node memory comparison results is shown in Fig.4. For VGG
architecture, even when employing the most memory-intensive VGG-19
architecture with a timestep of 20, the cumulative memory usage for all the
reversible SNN nodes within the entire network remains below 200MB. In
contrast, using conventional SNN nodes demands a substantial amount of memory,
up to 9032MB. For ResNet architectures, the ResNet-101 architecture with a
timestep of 10 needs about 28993MB using conventional SNN node, but only
1382MB using our reversible SNN node. As the number of model layers and the
timestep value increase, the memory savings achieved by our reversible SNN
node become more pronounced. Specifically, when utilizing the VGG-19
architecture with a timestep of 20, our reversible SNN node enjoys a
$\mathbf{58.65\times}$ memory reduction compared to the original SNN node. The
specific data values are shown in the Appendix.

Figure 4: Memory comparison between normal SNN node and our reversible SNN
node.

These experimental results align with our theoretical analysis in Section 3.1,
further validating that our design is able to significantly reduce memory
usage.

###  5.3 Training time Evaluation

To investigate the time efficiency of our designed backpropagation
architecture in comparison with the traditional reversible layer
backpropagation method, we employ two sets of backpropagation architectures
for our reversible SNN node. The first set utilizes the original reversible
layer backpropagation method, while the second set incorporates our newly
designed backpropagation architecture.

We employ VGG-11, VGG-13, VGG-16, and VGG-19 architectures with timesteps
ranging from 1 to 10. We compare the time required for one iteration of
training using the original SNN node, the reversible SNN node with the
original reversible layer backpropagation method, and the reversible SNN node
with our backpropagation architecture on the CIFAR-10 datasets. We perform all
the experiments on an empty RTX6000 GPU and keep the batch size = 64. The
reported times for each forward and backward pass are averages taken over all
iterations within the first five epochs.

Figure 5: Training time analysis. Solid lines: Backward process’s duration;
Dashed lines: Forward process’s duration; Red lines: Training time for the
original SNN; Green lines: Training time for the reversible SNN using the
original reversible layer backpropagation method; Blue lines: Training time
for the reversible SNN employing our proposed backpropagation architecture.

Fig.5 presents our measurement of the training time when the number of
timesteps is set to 4, 6, and 8. The forward computation times for the three
methods are virtually identical. The shortest backward processing time is
exhibited by the original SNN node, primarily due to it records all
intermediate values throughout the computation, thus eliminating the need for
recalculations. Comparatively, among the two reversible SNN nodes, our
backpropagation design achieves a $\mathbf{20\%}-\mathbf{30\%}$ increase in
speed over previous reversible layer backpropagation method during the
backward process. As the network expands, the superiority of our
backpropagation design becomes increasingly evident. Specifically, under the
VGG-19 architecture with a timestep of 8, our designed node is able to save
$\mathbf{23.8\%}$ of the total training time compared to the reversible node
using the original reversible layer backpropagation method. This aligns well
with our previous theoretical predictions in Section 4. Data for the other
timesteps is shown in the Appendix.

###  5.4 Ablation Study

Effects of parameters $\alpha$ and $\beta$ in our equations

In Eq.4 and Eq.5, we have two parameters: $\alpha$ and $\beta$. The optimal
setting for the parameter $\beta$ is 1, as this maximizes the preservation of
the original features of the data. We conduct experiments to assess the impact
of the $\alpha$ parameter on the model’s performance. We vary the $\alpha$
parameter from 0.05 to 0.8, and then employ architectures VGG-19, VGG-16,
VGG-13, and VGG-11 to evaluate the accuracy on the CIFAR100 dataset. The
results are shown on the left of Fig.6. We observe that varying $\alpha$
within the range of 0.05 to 0.8 impacts the final accuracy by approximately
1%. Generally, the model exhibits optimal performance when $\alpha$ is set
between 0.1 to 0.2.

Effects of number of groups for the various states

In Section 3.2, We introduce a method of splitting various input states into
two groups along the last dimension. Nonetheless, this method might encounter
issues under specific circumstances. For instance, if the last dimension of a
tensor is an odd number, it cannot be evenly divided into two groups. To
address this, we enhance the original algorithm: we divide the various input
states into $n$ groups according to the number of elements $n$ in the last
dimension. Eq.3-5 is then executed sequentially for each group. This
enhancement further improves the universality of our algorithm.

To evaluate the impact of the number of groups on the model, we modified part
of the fully connected layers in the original ResNet-19, ResNet-18, VGG-16,
VGG-13 network from 128 activations to 144 activations. This is to allow it to
have a wider variety of factors. We then evaluate the model’s performance with
the number of groups set to 2, 3, 6, 12, 24, 48, 72, and 144 respectively on
CIFAR100 dataset. The results are shown on the right of Fig.6. We observe that
the training accuracy improves as the number of groups increases. When the
number of groups approaches the number of elements $n$ in the last dimension,
the accuracy typically surpasses that of the original SNN node. This is
attributed to a larger number of groups yielding a higher fidelity
representation of the original data.

Figure 6: Left Figure: Test VGG-19,VGG-16,VGG-13,VGG-11 models on CIFAR100
dataset by using different $\alpha$ settings. Right Figure: Change activations
number from 128 to 144 for some fully connected layers inside ResNet-19,
ResNet-18, VGG-16, VGG-13 and test model performance for different number of
groups on CIFAR100. Rev.: Reversible SNN node. Ori.: Original SNN node. Mo.:
Modified network (Change some fully connected layers).

##  6 Conclusion and Discussion

This work addresses a fundamental bottleneck of current deep SNNs: their high
GPU memory consumption. We have designed a novel reversible SNN node that is
able to reduce memory complexity from $\mathcal{O}(n^{2})$ to
$\mathcal{O}(1)$. Specifically, our reversible SNN node allows our SNN network
to achieve $\mathbf{2.54}$ times greater memory efficiency than the current
SOTA SNN memory-efficient work on the CIFAR10 dataset, and $\mathbf{3.13}$
times greater on the CIFAR100 dataset. Furthermore, in order to tackle the
prolonged training time issue caused by the need for recalculating
intermediate values during backpropagation within our designed reversible SNN
node, we’ve innovated a new backpropagation approach specifically suited for
reversible architectures. This innovative method, when compared to the
original reversible layer architecture, achieves a substantial reduction in
overall training time by $\mathbf{23.7\%}$. As a result, we are able to train
over-parameterized networks that significantly outperform current models on
standard benchmarks while consuming less memory.

## References

  * [1] Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al.  Loihi: A neuromorphic manycore processor with on-chip learning.  Ieee Micro, 38(1):82–99, 2018.
  * [2] Mike Davies, Andreas Wild, Garrick Orchard, Yulia Sandamirskaya, Gabriel A Fonseca Guerra, Prasad Joshi, Philipp Plank, and Sumedh R Risbud.  Advancing neuromorphic computing with loihi: A survey of results and outlook.  Proceedings of the IEEE, 109(5):911–934, 2021.
  * [3] Peter U Diehl and Matthew Cook.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity.  Frontiers in computational neuroscience, 9:99, 2015.
  * [4] Laurent Dinh, David Krueger, and Yoshua Bengio.  Nice: Non-linear independent components estimation.  arXiv preprint arXiv:1410.8516, 2014.
  * [5] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio.  Density estimation using real nvp.  arXiv preprint arXiv:1605.08803, 2016.
  * [6] Haoqi Fan, Yanghao Li, Bo Xiong, Wan-Yen Lo, and Christoph Feichtenhofer.  Pyslowfast.  https://github.com/facebookresearch/slowfast, 2020.
  * [7] Wei Fang and et.al.  Deep residual learning in spiking neural networks.  NeurIPS, 2021.
  * [8] Aidan N Gomez, Mengye Ren, Raquel Urtasun, and Roger B Grosse.  The reversible residual network: Backpropagation without storing activations.  Advances in neural information processing systems, 30, 2017.
  * [9] Yufei Guo, Weihang Peng, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Xuhui Huang, and Zhe Ma.  Joint a-snn: Joint training of artificial and spiking neural networks via self-distillation and weight factorization.  Pattern Recognition, page 109639, 2023.
  * [10] Bing Han and Kaushik Roy.  Deep spiking neural network: Energy efficiency through time based coding.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X, pages 388–404. Springer, 2020.
  * [11] Jun-Jie Huang and Pier Luigi Dragotti.  Winnet: Wavelet-inspired invertible network for image denoising.  IEEE Transactions on Image Processing, 31:4377–4392, 2022.
  * [12] Shaoyi Huang, Haowen Fang, Kaleel Mahmood, Bowen Lei, Nuo Xu, Bin Lei, Yue Sun, Dongkuan Xu, Wujie Wen, and Caiwen Ding.  Neurogenesis dynamics-inspired spiking neural network training acceleration.  arXiv preprint arXiv:2304.12214, 2023.
  * [13] Youngeun Kim, Yuhang Li, Hyoungseob Park, Yeshwanth Venkatesha, Ruokai Yin, and Priyadarshini Panda.  Lottery ticket hypothesis for spiking neural networks.  arXiv preprint arXiv:2207.01382, 2022.
  * [14] Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya.  Reformer: The efficient transformer.  arXiv preprint arXiv:2001.04451, 2020.
  * [15] Guohao Li, Matthias Müller, Bernard Ghanem, and Vladlen Koltun.  Training graph neural networks with 1000 layers.  In International conference on machine learning, pages 6437–6449. PMLR, 2021.
  * [16] Kang Liu, Dong Liu, Li Li, Ning Yan, and Houqiang Li.  Semantics-to-signal scalable image compression with learned revertible representations.  International Journal of Computer Vision, 129(9):2605–2621, 2021\.
  * [17] Matthew MacKay, Paul Vicol, Jimmy Ba, and Roger B Grosse.  Reversible recurrent neural networks.  Advances in Neural Information Processing Systems, 31, 2018.
  * [18] Karttikeya Mangalam, Haoqi Fan, Yanghao Li, Chao-Yuan Wu, Bo Xiong, Christoph Feichtenhofer, and Jitendra Malik.  Reversible vision transformers.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10830–10840, 2022.
  * [19] Emre O Neftci, Hesham Mostafa, and Friedemann Zenke.  Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks.  IEEE Signal Processing Magazine, 36(6):51–63, 2019.
  * [20] Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, and Dennis DeCoste.  Memory efficient 3d u-net with reversible mobile inverted bottlenecks for brain tumor segmentation.  In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 6th International Workshop, BrainLes 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Revised Selected Papers, Part II, pages 388–397. Springer, 2021.
  * [21] Rachmad Vidya Wicaksana Putra and Muhammad Shafique.  Q-spinn: A framework for quantizing spiking neural networks.  In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2021.
  * [22] Nitin Rathi, Priyadarshini Panda, and Kaushik Roy.  Stdp-based pruning of connections and weight quantization in spiking neural networks for energy-efficient recognition.  IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 38(4):668–677, 2018.
  * [23] Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy.  Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation.  In International Conference on Learning Representations, 2020.
  * [24] Bodo Rueckauer, Connor Bybee, Ralf Goettsche, Yashwardhan Singh, Joyesh Mishra, and Andreas Wild.  Nxtf: An api and compiler for deep spiking neural networks on intel loihi.  ACM Journal on Emerging Technologies in Computing Systems (JETC), 18(3):1–22, 2022.
  * [25] Amar Shrestha, Haowen Fang, Zaidao Mei, Daniel Patrick Rider, Qing Wu, and Qinru Qiu.  A survey on neuromorphic computing: Models and hardware.  IEEE Circuits and Systems Magazine, 22(2):6–35, 2022.
  * [26] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [27] Sonali Singh, Anup Sarma, Sen Lu, Abhronil Sengupta, Mahmut T Kandemir, Emre Neftci, Vijaykrishnan Narayanan, and Chita R Das.  Skipper: Enabling efficient snn training through activation-checkpointing and time-skipping.  In 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 565–581. IEEE, 2022.
  * [28] Jianshi Tang, Fang Yuan, Xinke Shen, Zhongrui Wang, Mingyi Rao, Yuanyuan He, Yuhao Sun, Xinyi Li, Wenbin Zhang, Yijun Li, et al.  Bridging biological and artificial neural networks with emerging neuromorphic devices: fundamentals, progress, and challenges.  Advanced Materials, 31(49):1902761, 2019.
  * [29] Jianxiong Tang, Jianhuang Lai, Xiaohua Xie, Lingxiao Yang, and Wei-Shi Zheng.  Snn2ann: A fast and memory-efficient training framework for spiking neural networks.  arXiv preprint arXiv:2206.09449, 2022.
  * [30] THUDM.  Cogdl: An extensive toolkit for deep learning on graphs.  https://github.com/THUDM/cogdl, 2023.
  * [31] Jibin Wu, Yansong Chua, Malu Zhang, Guoqi Li, Haizhou Li, and Kay Chen Tan.  A tandem learning rule for effective training and rapid inference of deep spiking neural networks.  IEEE Transactions on Neural Networks and Learning Systems, 2021\.
  * [32] Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin.  Online training through time for spiking neural networks.  arXiv preprint arXiv:2210.04195, 2022.
  * [33] Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Yisen Wang, and Zhouchen Lin.  Training feedback spiking neural networks by implicit differentiation on the equilibrium state.  Advances in Neural Information Processing Systems, 34:14516–14528, 2021.


* (189) Huang, X., Yang, L.: A hybrid genetic algorithm for multi objective flexible job shop scheduling problem considering transportation time. International Journal of Intelligent Computing and Cybernetics 12(2), 154–174 (2019). https://doi.org/10.1108/IJICC-10-2018-0136
  * (190) Ojstersek, R., Zhang, H., Liu, S., Buchmeister, B.: Improved heuristic kalman algorithm for solving multi-objective flexible job-shop scheduling problem. Procedia Manufacturing 17, 895–902 (2018). https://doi.org/10.1016/j.promfg.2018.10.142
  * (191) Zhou, Y., Yang, J., Zheng, L.: Multi-agent based hyper-heuristics for multi-objective flexible job-shopscheduling: A case study in an aero-engine blade manufacturing plant. IEEE Access 7, 21147–21176 (2019). https://doi.org/10.1109/ACCESS.2019.2897603
  * (192) Seyyedhasani, H., Peng, C., Jang, W.-j., Vougioukas, S.G.: Collaboration of human pickers and crop-transporting robots during harvesting – part i: Model and simulator development. Computers and Electronics in Agriculture 172 (2020). https://doi.org/10.1016/j.compag.2020.105324
  * (193) Conesa-Muñoz, J., Bengochea-Guevara, J.M., Andujar, D., Ribeiro, A.: Route planning for agricultural tasks: A general approach for fleets of autonomous vehicles in site-specific herbicide applications. Computers and Electronics in Agriculture 127, 204–220 (2016). https://doi.org/10.1016/j.compag.2016.06.012
  * (194) Edwards, G., Sorensen, C.G., Bochtis, D., Munkholm, L.J.: Optimized schedules for sequential agricultural operations using a tabu search method. Computers and Electronics in Agriculture 117, 102–113 (2015). https://doi.org/10.1016/j.compag.2015.07.007
  * (195) Ahsan, Z., Dankowicz, H.: Optimal scheduling and sequencing for large-scale seeding operations. Computers and Electronics in Agriculture 163, 104728 (2019). https://doi.org/10.1016/j.compag.2019.01.052
  * (196) Jensen, M.F., Bochtis, D., Sorensen, C.G.: Coverage planning for capacitated field operations, part ii: Optimisation. Biosystems Engineering 139, 149–164 (2015). https://doi.org/10.1016/j.biosystemseng.2015.07.002
  * (197) Santoro, E., Soler, E.M., Cherri, A.C.: Route optimization in mechanized sugarcane harvesting. Computers and Electronics in Agriculture 141, 140–146 (2017). https://doi.org/10.1016/j.compag.2017.07.013
  * (198) Cheein, F.A., Torres-Torriti, M., Hopfenblatt, N.B., Prado, A.J., Calabi, D.: Agricultural service unit motion planning under harvesting scheduling and terrain constraints. Journal of Field robotics 34, 1531–1542 (2017). https://doi.org/10.1002/rob.21738
  * (199) Richards, D., Patten, T., Fitch, R.C., Ball, D., Sukkarieh, S.: User interface and coverage planner for agricultural robotics. Australasian Conference on Robotics and Automation (2015)
  * (200) Mann, M.P., Zion, B., Shmulevich, I., Rubinstein, D., Linker, R.: Combinatorial optimization and performance analysis of a multi-arm cartesian robotic fruit harvester–extensions of graph coloring. Journal of Intelligent & Robotic Systems 82(2-4), 399–411 (2016). https://doi.org/10.1007/s10846-015-0211-5
  * (201) Barnett, J., Duke, M., Au, C.K., Lim, S.H.: Work distribution of multiple cartesian robot arms for kiwifruit harvesting. Computers and Electronics in Agriculture 169, 105202 (2020). https://doi.org/10.1016/j.compag.2019.105202
  * (202) Kurtser, P., Edan, Y.: Planning the sequence of tasks for harvesting robots. Robotics and Autonomous Systems 131 (2020). https://doi.org/10.1016/j.robot.2020.103591
  * (203) Miller, A.T., Allen, P.K.: Examples of 3d grasp quality computations 2, 1240–1246 (1999). https://doi.org/10.1109/ROBOT.1999.772531
  * (204) Rodríguez, F., Moreno, J.C., Sánchez, J.A., Berenguel, M.: Grasping in agriculture: State-of-the-art and main characteristics. Mechanisms and Machine Science 10, 385–409 (2012). https://doi.org/10.1007/978-1-4471-4664-3_15
  * (205) Mu, L., Cui, G., Liu, Y., Cui, Y., Fu, L., Gejima, Y.: Design and simulation of an integrated end-effector for picking kiwifruit by robot. Information Processing in Agriculture 7(1), 58–71 (2020). https://doi.org/10.1016/j.inpa.2019.05.004
  * (206) Liu, J., Li, P., Li, Z.: A multi-sensory end-effector for spherical fruit harvesting robot, 258–262 (2007). https://doi.org/10.1109/ICAL.2007.4338567
  * (207) Jia, B., Zhu, A., Yang, S.X., Mittal, G.S.: Integrated gripper and cutter in a mobile robotic system for harvesting greenhouse products, 1778–1783 (2009). https://doi.org/10.1109/ROBIO.2009.5420430
  * (208) Dimeas, F., Sako, D.V., Moulianitis, V., Aspragathos, N.: Design and fuzzy control of a robotic gripper for efficient strawberry harvesting. Robotica 33(5), 1085–1098 (2014). https://doi.org/10.1017/S0263574714001155
  * (209) Zhong, H., Nof, S.Y., Berman, S.: Asynchronous cooperation requirement planning with reconfigurable end-effectors. Robotics and Computer Integrated Manufacturing 34, 95–104 (2015). https://doi.org/10.1016/j.rcim.2014.11.004
  * (210) Pedrazzoli, P., Rinaldi, R., Boer, C.R.: A rule based approach to the gripper selection issue for the assembly process, 202–207 (2001). https://doi.org/10.1109/ISATP.2001.928990
  * (211) Pham, D.T., Gourashi, N.S., Eldukhri, E.E.: Automated configuration of gripper systems for assembly tasks. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture 221, 1643–1649 (2007). https://doi.org/10.1243/09544054JEM878SC
  * (212) Sanfilippo, F., Salvietti, G., Zhang, H.X., Hildre, H.P., Prattichizzo, D.: Efficient modular grasping: An iterative approach, 1281–1286 (2012). https://doi.org/10.1109/BioRob.2012.6290693
  * (213) Brown, R.G., Brost, R.C.: A 3d modular gripper design tool 3, 2332–2339 (1997). https://doi.org/10.1109/ROBOT.1997.619310
  * (214) Balan, L., Bone, G.M.: Automated gripper jaw design and grasp planning for sets of 3d objects. Journal of Field Robotics 20, 147–162 (2003). https://doi.org/10.1002/rob.10076
  * (215) Velasco, V.B., Newman, W.S.: An approach to automated gripper customization using rapid prototyping technology (1996)
  * (216) Velasco, V.B., Newman, W.S.: Computer-assisted gripper and fixture customization using rapid-prototyping technology 4, 3658–3664 (1998). https://doi.org/10.1109/ROBOT.1998.681393
  * (217) Honarpardaz, M., Tarkian, M., Feng, X., Sirkett, D., Ölvander, J.: Generic automated finger design 5, 1–9 (2016). https://doi.org/10.1115/DETC2016-60514
  * (218) Sahbani, A., El-Khoury, S., Bidaud, P.: An overview of 3d object grasp synthesis algorithms. Robotics and Autonomous Systems 60(3), 326–336 (2012). https://doi.org/10.1016/j.robot.2011.07.016
  * (219) Bicchi, A., Kumar, V.: Robotic Grasping and Contact: a Review vol. 1, pp. 348–353 (2000). https://doi.org/10.1109/ROBOT.2000.844081
  * (220) Li, J.-W., Liu, H., Cai, H.-G.: On computing three-finger force-closure grasps of 2-d and 3-d objects. IEEE Transactions on Robotics and Automation 19(1), 155–161 (2003). https://doi.org/10.1109/TRA.2002.806774
  * (221) Han, L., Trinkle, J.C., Li, Z.: Grasp analysis as linear matrix inequality problems, 1261–1268 (1999). https://doi.org/10.1109/ROBOT.1999.772534
  * (222) Mishra, B., Schwartz, J.T., Sharir, M.: On the existence and synthesis of multifinger positive grips. Algorithmica 2(1), 541–558 (1987). https://doi.org/10.1007/BF01840373
  * (223) Liu, Y.-H.: Qualitative test and force optimization of 3-d frictional form-closure grasps using linear programming. IEEE Transactions on Robotics and Automation 15(1), 163–173 (1999). https://doi.org/10.1109/70.744611
  * (224) Borst, C., Fischer, M., Hirzinger, G.: Grasping the dice by dicing the grasp 3, 3692–3697 (2003). https://doi.org/10.1109/IROS.2003.1249729
  * (225) Miller, A.T., Knoop, S., Christensen, H.I., Allen, P.K.: Automatic grasp planning using shape primitives 2, 1824–1829 (2003). https://doi.org/10.1109/ROBOT.2003.1241860
  * (226) Ding, D., Liu, Y.-H., Wang, S.: Computing 3-d optimal form-closure grasps 4, 3573–3578 (2000). https://doi.org/10.1109/ROBOT.2000.845288
  * (227) Ding, D., Liu, Y.-H., Wang, M.Y.: On computing immobilizing grasps of 3-d curved objects. Proceedings 2001 IEEE International Symposium on Computational Intelligence in Robotics and Automation, 11–16 (2001). https://doi.org/10.1109/CIRA.2001.1013165
  * (228) Liu, Y.-H., Lam, M.-L., Ding, D.: A complete and efficient algorithm for searching 3-d form-closure grasps in the discrete domain. IEEE Transactions on Robotics 20(5), 805–816 (2004). https://doi.org/10.1109/TRO.2004.829500
  * (229) Eizicovits, D., Berman, S.: Efficient sensory-grounded grasp pose quality mapping for gripper design and online grasp planning. Robotics and Autonomous Systems 62(8), 1208–1219 (2014). https://doi.org/10.1016/j.robot.2014.03.011
  * (230) Liu, S., Carpin, S.: Global grasp planning using triangular meshes, 4904–4910 (2015). https://doi.org/10.1109/ICRA.2015.7139880
  * (231) Hemming, J., Bac, C.W., Tuijl, B., Barth, R., Bontsema, J., Pekkeriet, E., Henten, E.V.: A robot for harvesting sweet-pepper in greenhouses (2014)
  * (232) Bohg, J., Morales, A., Asfour, T., Kragic, D.: Data-driven grasp synthesis - a survey. IEEE Transactions on Robotics 30(2), 289–309 (2014). https://doi.org/%****␣main.bbl␣Line␣3850␣****10.1109/TRO.2013.2289018
  * (233) Kim, J., Iwamoto, K., Kuffner, J.J., Ota, Y., Pollard, N.S.: Physically based grasp quality evaluation under pose uncertainty. IEEE Transactions on Robotics 29(6), 1424–1439 (2013). https://doi.org/10.1109/TRO.2013.2273846
  * (234) Wolniakowski, A., Miatliuk, K., Kruger, N., Rytz, J.A.: Automatic evaluation of task-focused parallel jaw gripper design. International Conference on Simulation, Modeling, and Programming for Autonomous Robots, 450–461 (2014). https://doi.org/10.1007/978-3-319-11900-7_38
  * (235) Fernández, R., Salinas, C., Montes, H., Sarria, J.: Multisensory system for fruit harvesting robots. experimental testing in natural scenarios and with different kinds of crops. Sensors 14(12), 23885–23904 (2014). https://doi.org/%****␣main.bbl␣Line␣3900␣****10.3390/s141223885
  * (236) Bac, C.W., Hemming, J., van Tuijl, B.A.J., Barth, R., Wais, E., van Henten, E.J.: Performance evaluation of a harvesting robot for sweet pepper. Journal of Field Robotics 34(6), 1123–1139 (2017). https://doi.org/10.1002/rob.21709
  * (237) Tardáguila, J., Blasco, J., Diago, M.P.: Vinerobot: A new robot for vineyard monitoring using non-invasive sensing technologies. 9th International Cool Climate Wine Symposium (2016). Retrieved from https://digital.csic.es/handle/10261/148399 \- Last accessed: 19-05-2022
  * (238) Fernández-Novales, J., Garde-Cerdán, T., Tardáguila, J., Gutiérrez-Gamboa, G., Pérez-Álvarez, E.P., Diago, M.P.: Assessment of amino acids and total soluble solids in intact grape berries using contactless vis and nir spectroscopy during ripening. Talanta 199, 244–253 (2019). https://doi.org/10.1016/j.talanta.2019.02.037
  * (239) Aquino, A., Millan, B., Diago, M.P., Tardaguila, J.: Automated early yield prediction in vineyards from on-the-go image acquisition. Computers and Electronics in Agriculture 144, 26–36 (2018). https://doi.org/10.1016/j.compag.2017.11.026
  * (240) Fernández-Novales, J., Tardaguila, J., Gutiérrez, S., Marañón, M., Diago, M.P.: In field quantification and discrimination of different vineyard water regimes by on-the-go nir spectroscopy. Biosystems Engineering 165, 47–58 (2018). https://doi.org/10.1016/j.biosystemseng.2017.08.018
  * (241) Lopes, C.M., Graça, J.D., Sastre, J., Reyes, M., Guzman, R., Braga, R., Monteiro, A., Pinto, P.A.: Vineyard yield estimation by vinbot robot - preliminary results with the white variety viosinho (2016). https://doi.org/10.13140/RG.2.1.3912.0886
  * (242) Lopes, C.M., Torres, A., Guzman, R., Graça, J.D., Monteiro, A., Braga, R.P., Barriguinha, A., Victorino, G., Reys, M.: Using an unmanned ground vehicle to scout vineyards for non-intrusive estimation of canopy features and grape yield. 20th GiESCO International Meeting (2017)
  * (243) Astolfi, P., Gabrielli, A., Bascetta, L., Matteucci, M.: Vineyard autonomous navigation in the echord++ grape experiment. IFAC-PapersOnLine 51(11), 704–709 (2018). https://doi.org/10.1016/j.ifacol.2018.08.401
  * (244) Leu, A., Razavi, M., Langstädtler, L., Ristić-Durrant, D., Raffel, H., Schenck, C., Gräser, A., Kuhfuss, B.: Robotic green asparagus selective harvesting. IEEE/ASME Transactions on Mechatronics 22(6), 2401–2410 (2017). https://doi.org/10.1109/TMECH.2017.2735861
  * (245) Fernández-Novales, J., J.Tardáguila, Gutiérrez, S., Diago, M.P.: On-the-go vis $+$ sw $-$ nir spectroscopy as a reliable monitoring tool for grape composition within the vineyard. Molecules 24(15), 2795 (2019). https://doi.org/10.3390/molecules24152795
  * (246) Gutiérrez, S., Tardaguila, J., Fernández-Novales, J., Diago, M.P.: On-the-go hyperspectral imaging for the in-field estimation of grape berry soluble solids and anthocyanin concentration. Australian Journal of Grape and Wine Research 25(1), 127–133 (2019). https://doi.org/10.1111/ajgw.12376
  * (247) Gutiérrez, S., Diago, M.P., Fernández-Novales, J., Tardaguila, J.: On-the-go thermal imaging for water status assessment in commercial vineyards. Advances in Animal Biosciences 8(2), 520–524 (2017). https://doi.org/10.1017/S204047001700108X
  * (248) Grimstad, L., Pham, C.D., Phan, H.T., From, P.J.: On the design of a low-cost, light-weight, and highly versatile agricultural robot. IEEE Workshop on Advanced Robotics and its Social Impacts, ARSO (2016). https://doi.org/10.1109/ARSO.2015.7428210
  * (249) Harvest Croo Robotics. https://harvestcroo.com/ \- Last accessed: 22-11-2022
  * (250) Dogtooth. https://dogtooth.tech/ \- Last accessed: 22-11-2022
  * (251) Agrobot E-Series. http://agrobot.com \- Last accessed: 22-11-2022
  * (252) OCTINION. http://octinion.com/products/agricultural-robotics/ rubion \- Last accessed: 22-11-2022
  * (253) SAGA Robotics. https://sagarobotics.com \- Last accessed: 22-11-2022
  * (254) Grimstad, L., From, P.J.: The thorvald ii agricultural robotic system. Robotics 6(4) (2017). https://doi.org/10.3390/robotics6040024
  * (255) MetoMotion. https://metomotion.com \- Last accessed: 22-11-2022
  * (256) Root-AI. https://www.appharvest.com/press_release/appharvest-acquires-agricultural-robotics-and-artificial-intelligence-company-root-ai-to-increase-efficiency/ \- Last accessed: 22-11-2022
  * (257) AppHarvest. https://www.appharvest.com/ \- Last accessed: 22-11-2022
  * (258) ENERGID. https://www.energid.com/industries/agricultural-robotics \- Last accessed: 22-11-2022
  * (259) VISION ROBOTICS. https://www.visionrobotics.com/vr-grapevine -pruner \- Last accessed: 22-11-2022
  * (260) naio Technologies. https://www.naio-technologies.com/en/ted/ \- Last accessed: 22-11-2022
  * (261) ViTiBOT. https://vitibot.fr/en \- Last accessed: 22-11-2022