# Deep Learning combined with singular value decomposition to reconstruct
databases in fluid dynamics

P. Díaz1, A. Corrochano1,
M. López-Martín2 and S. Le Clainche1 1 E.T.S.I. Aeronáutica y del
Espacio,Universidad Politécnica de Madrid, Spain  2 E.T.S.I.T., Universidad de
Valladolid, Valladolid, Spain

###### Abstract

Fluid Dynamics problems are characterized by being multidimensional and
nonlinear. Therefore, experiments and numerical simulations are complex and
time-consuming. Motivated by this, the need arises to find new techniques to
obtain data in a simpler way and in less time. In this article, we present a
novel methodology based on physical principles to reconstruct three-, four-
and five-dimensional databases from a strongly sparse sensors as input. The
methodology consists of combining Single Value Decomposition (SVD) with neural
networks. The neural network used is characterized by a simple architecture
based on combining two autoencoders that work in parallel and are joined in
the last layer. This new algorithm has been proved with three databases with
different dimensions and complexities: in an Atmospheric Boundary Layer (ABL)
with a turbulence model and in the flow past a two- and a three-dimensional
cylinder. Summarizing, this work proposes a new hybrid physics-based machine
learning model with a simple, robust and generalizable architecture, which
allows reconstructing databases from very few sensors and with a very low
computational cost.

###### keywords:

Reduced order models , Deep learning architectures , SVD , Modal decomposition
, Neural networks , Fluid Dynamics

††journal: Journal of Preprint

##  1 Introduction

Nowadays, the aeronautical industry is closely linked to computational fluid
dynamics due to the need to create models that simulate the behavior of the
aircraft [1] to create new designs less pollutant and more energetically
efficient. Certain test necessary to obtain high-fidelity databases are
difficult to carry out, require a lot of time and are also expensive. For
example, one of the most challenging areas of any aircraft design is the
prediction of the aerodynamic coefficients present before, during and after
stall of the aircraft [2]. To obtain such coefficients multiple wind tunnel
[3] and flight tests [4] are necessary, which are not always feasible due to
the increased cost and the requirement of sophisticated infrastructures.
Another example where obtaining data through experimental tests is complicated
is in acoustic tests, which are motivated by the need to reduce the noise
produced by aircraft on takeoff and landing. This type of testing requires
hundreds of microphones to measure noise and thousands of meters of cables
[5], resulting in a bunch of sparse measurements, sometimes difficult to
interpret.

Because of these difficulties, there is a growing interest in mathematical
methods to represent as realistically as possible the behavior of aircraft
with the least number of test as possible [6]. In this way, experimental data
would be combined with predicted or simulated data to correct the design prior
to the flight test program [7]. In addition to the difficulty of experimental
configurations, in many cases, the data to be predicted or simulated have to
do with the behavior of fluids around certain bodies. These flows are usually
highly-dimensional complex flows (in transient and turbulent regime), and
their prediction is particularly difficult, since it requires not only a rich
learning model, but also a suitable mapping function that generates an
intermediate representation in a low-dimensional feature space (latent space)
to reduce its computational needs.

Numerical simulations and experiments are the two methodologies to study in
detail complex flows in fluid dynamics problems. But high-fidelity simulations
are expensive and experiments sometimes provide unresolved databases.
Therefore, it is complicate to understand the flow physics related to specific
problems. Hence, there is a spreading interest in obtaining a method that
allows to enhance the resolution of databases, or event to reconstruct two- or
three-dimensional flow fields from sparse databases with a low computational
cost, which would be an advance for existing problems in fluid mechanics.

There is a growing need to find techniques capable of augmenting databases
without the need for experimental setups and heavy simulations [8]. That is
why the main objective of this project is to develop a methodology capable of
reconstructing databases.

In the line of research on database reconstruction, there are some previous
works. For instance, in Ref. [9], a neural network called RAndomly-SEEDed
super-resolution GAN (RaSeedGAN) capable of switching from a low resolution
image to a high resolution image is presented. This network is tested with
turbulent fluid and sea temperature. Also, in Ref. [10] an unsupervised GAN
neural network is tested and compared with the results of different supervised
GAN and Convolutional neural networks. Another type of neural network
architecture used to reconstruct flow fields is convolutional neural network
[11][12]. In Ref. [13], they proposed a shallow decoder (SD) and the input
data of the SD are catching selecting random sensors on the surface of a
cylinder.

In line with the above-mentioned studies, this methodology must be able to be
applied to different problems regardless of the nature of the data. This
implies that the results must be obtained without using the equations that
define the fluid behavior and that it is necessary to define generalizable
architectures. For this purpose, we have designed a new methodology with
neural networks and, keeping the same network architecture, we have tested its
robutness and generalization capabilities with different databases. The main
difference between the method proposed in this paper and the other existing
methods lies in the network architecture. The architecture of the neural
network has been specifically designed to be simple and efficient, being a
design guideline to have a network with as few parameters (weights) as
possible to reduce the complexity of the model and the possibility of
overfitting, considering the small amount of data available to train the
model. Additionally, the architecture must address noisy data and provide
representation learning capabilities that are always crucial in any deep
learning model [14]. That is the reason for proposing a parallel deep learning
architecture based on autoencoders with a final integration layer. The
proposed architecture is lightweight and simple and can be trained with a
relatively small amount of data with a remarkable ability to generalize under
different flow dynamics.

Furthermore, when using machine learning tools there is a computational limit
[15] because the computers to be used for programming will have limited
temporary memory. This means that the algorithm must have a low computational
cost. Therefore, we propose a novel, simple, robust and generalizable
architecture to reconstruct databases in fluid dynamics.

The current work is organized as follows. Section 2 explains the application
of this paper in fluid mechanics and Section 3 is dedicated to the preliminary
data preparation. In Section 4, the methodology and architecture of the neural
network are explained. In Section 5, the most relevant results are shown for
different databases. In addition, error tables are included in this section.
Finally, section 6 includes the conclusions.

##  2 Application to Fluid Dynamics

For the development of this research, data from three different complex fluid
fields have been used: air in the atmospheric boundary layer (ABL), two- and
three-dimensional flow past a circular cylinder. The three databases have been
generated numerically. The equations governing fluid dynamics problems are the
Navier-Stokes equations. These equations for a viscous, incompressible and
Newtonian flow are as follows:

$\nabla\cdot\overrightarrow{V}=0$ (1)

$\frac{\partial v}{\partial
t}+\left(\overrightarrow{V}\cdot\nabla\right)\overrightarrow{V}=-\nabla
p+\frac{1}{Re}\vartriangle\overrightarrow{V}$ (2)

where $\overrightarrow{V}$ and $p$ are the velocity vector and the pressure,
respectively. $Re$ is the Reynolds number defined as $Re=\rho UL/\mu$ , where
$U$ is the characteristic velocity, $L$ is the characteristic lenght, $\rho$
is the density of the fluid and $\mu$ is the dynamic viscosity of the fluid.
The equations are non-dimensioned with characteristic length and
characteristic time, $L$ and $L/U$.

Additionally, for the atmospheric boundary layer database it is necessary to
add to the above two equations the energy conservation equation defined as:

$\rho c_{p}\left[\frac{\partial T}{\partial
t}+\left(\overrightarrow{V}\cdot\nabla\right)T\right]=k\nabla^{2}T+\phi$ (3)

where $T$ is the temperature of the fluid, $c_{p}$ is the specific heat, $k$
is the thermal conductivity coefficient and $\phi$ is the dissipation term.

###  2.1 Athmospheric Boundary Layer (ABL) database

A different and relevant problems related to complex flows and with
experimental difficulties is the characterization of the air flow in the
atmospheric boundary layer (ABL) [16, 17]. This part of the atmosphere is
located within the troposphere and occupies approximately 1 km from the
earth’s surface, see Fig. 1. It is of vital importance to know the
characteristics of the air in this area because this is where human life
develops, where almost all meteorological phenomena are created and where
water vapor, aerosols, pollution, among others, are mixed. Therefore, the flow
in the atmospheric boundary layer is highly turbulent and variable, which is
why it is challenging to characterize and predict the state variables, such as
pressure and temperature, of the air in this area. Understanding the flow
behavior can provide great advances in studies related to urban flows,
pollutant dispersion and many more applications.

Figure 1:  Sketch of where the atmospheric boundary layer is located in the
atmosphere.

The database is reduced to a small number of points (extracted from the total
grid), modelling in this way a set of measurements collected from sensor
networks distributed along the terrain. The atmospheric boundary layer is
characterized by a turbulent flow that varies according to a large number of
variables such as weather conditions, vegetation and buildings in the study
area. The variables measured the present study includes velocity vector,
pressure and temperature fields.

To obtain the database, numerical simulations have been performed. The
turbulent flow has been modelled using Reynolds Averaged Navier Stokes (RANS)
k-$\epsilon$ turbulence model, which is commonly used to model the atmospheric
boundary layer. The database has been generated using the open source software
OpenFoam and has been validated with experiments and simulations. Wall
functions have been used to properly model the ABL. More information about the
model and the obtained data can be found in Ref. [18].

The length of the domain is 5000 m (x-direction) to ensure a developed flow,
and the width is limited to 500 m (y-direction), as presented in the
literature [18]. The dimension of the ABL database is indicated in Tab. 1,
where the three variables correspond to streamwise velocity, pressure and
temperature of the air in the atmospheric boundary layer. Furthermore, since
the data are averaged in time (RANS model), there is not dimension related to
the temporal variable.

###  2.2 Circular Cylinder databases

The analysis of the flow behavior around a circular cylinder is a problem used
as a benchmark to validate methodologies. The dynamics of the cylinder is
closely linked to the concept of the Re number defined as $Re=\rho UD/\mu$,
where $D$ is the diameter of the cylinder and the rest of the variables were
defined before. The flow is steady for low Re numbers. From $Re\thicksim 46$,
we start to see how a Hofp bifurcation produces an unsteady flow which is lead
by a von Karman vortex street [19]. The oscillations remain bidimensional
until $Re\thicksim 189$ where a second bifurcation occurs and the flow becomes
three-dimensional for some specific wavelengths in the spanwise direction. For
high Re values, the flow becomes turbulent and hence fully three-dimensional
[20].

This behavior is defined by the Navier-Stokes equations described above (Eq. 1
and 2). In this research two numerical simulations have been analysed, the
two-dimensional cylinder at $Re=100$ and the tree-dimensional cilinder at
$Re=280$. In both cases, numerical simulations have been carried out with the
domains defined in the literature [20] using the open source solver Nek5000 to
solve the Navier-Stokes equations. The boundary conditions set in the
simulation are Dirichlet for velocity ($v_{x}=v_{y}=v_{z}=0$) and Neumann for
pressure. The conditions in the inlet and outlet of the domain are the same:
$v_{x}=1$, $v_{y}=v_{z}=0$ and Neumann condition for pressure. The domain of
the computational simulations is composed by 600 rectangular elements, each
one discretized using the polynomial order $\Pi=9$. The dimensions of the
computational domain non-dimensionalized with the diameter of the cylinder.
The size of the domain in the normal direction is $L_{y}=\textpm 15D$, and in
the streamwise direction are $L_{x}=15D$ upstream of the cylinder and $Lx=50D$
downstream of the cylinder. Details about the database used in this article
can be found in Ref. [21].

  * •

Two-dimensional Cylinder database (Cyl 2D)

This database represents the saturated flow around a 2D cylinder with Reynolds
number $Re=100$ [21]. The database analysed is formed by $150$ snapshots,
equispaced in time with an interval of $0.2$. The dimension of the two-
dimensional cylinder database is indicated in Tab. 1. The three variables used
in the reconstruction of the database correspond to the two components of the
velocity vector $\boldsymbol{V}=(u,v)$ (streamwise and normal velocity) and
the vorticity
$\overrightarrow{\omega}=\overrightarrow{\nabla}\times\overrightarrow{\boldsymbol{V}}$.

  * •

Three-dimensional Cylinder database (Cyl 3D)

This database represents the value of the velocity of the flow past a circular
cylinder at $Re=280$ [22]. A set of 599 snapshots had been collected from the
beginning of the numerical simulations equispaced in time with an interval of
1. However, the spanwise velocity does not start to develop until time $350$,
and the simulation does not converge (saturated regime) until time $500$ [23].
Because of this, only the last $299$ snapshots have been chosen as the
representation of the saturated flow regime for the entire database. The
dimension of the three-dimensional cylinder database is in Tab. 1. Regarding
this database, the three variables represent the three components of the
velocity (u,v,w). In the three-dimensional flow field, the database has one
more dimension than the rest of the databases.

Table 1: Summary of the dimensions of the databases. $N_{var}$ represents the
number of variables. $N_{x}$, $N_{y}$ and $N_{z}$ represent the number of
points in x, y and z direction, respectively. $N_{t}$ represents the number of
snapshots.

Database | $N_{t}$ | $N_{y}$ | $N_{x}$ | $N_{z}$ | $N_{var}$
---|---|---|---|---|---
ABL | - | 501 | 2001 | - | 3
Cyl 2D | 151 | 449 | 199 | - | 3
Cyl 3D | 299 | 100 | 40 | 64 | 3

##  3 Data preparation

Before applying the methodology, each database is organized as if the data
were collected by a group of sensors (with reduced information). Then, the new
methods is applied to reconstruct the original database. The results are
compared with the original solution.

The reduction of the databases has been performed in an homogeneous way, i.e.,
$1$ out of each $k$ points have been chosen in each dimension of the space in
such a way that all points are equispaced with respect to each other.
Additionally, for the ABL database, the performance of the method has also
been tested when only a few points have been chosen in a non-homogeneous way,
for instance, concentrated in the wall of the domain (where the strongest
changes of the flow occur).

In order to better clarify what is done at this step, an example of
homogeneous downsampling with the 2D Cylinder database is shown in Fig. 2. In
the first plot we have the original representation of the streamwise velocity.
By downsampling, a scale of type 1:k is chosen. In the case of Fig. 2, a
downsampling scale of 1:50 has been chosen, i.e. 1 point out of 50 points is
chosen. Since this basis consisted of 449 points in $x$ and 199 points in $y$,
the result of reducing the basis by 1:50 gives 9 points in $x$ and 4 points in
$y$. The representation of the reduced matrix of $9\times 4$ is the one shown
in the rightmost graph of Fig. 2.

Figure 2:  Example of downsampling to model sensor measurements. Data
downsampled equidistant taking 1 every 50 points. Dimension of the original
and reduced matrix presentations are $449\times 199$ and $9\times 4$ grid
points, respectively.

Once the reduction of points in the databases has been performed, it is
necessary to reorganize the data before applying the methodology. The spatial
dimensions should be collected in a matrix called a spatial matrix. If the
database has only x and y axis, the columns of the spatial matrix will
correspond to one axis and the rows to another axis. In case the database is
three dimensional in space (i.e. it has points on the three axes x,y,z), two
axes must be joined by means of a reshaped dimension. Then, there will be a
spatial matrix for each time instant which will generate the different
snapshots. This structure (shown in 3) is repeated for each variable. In case
there is no temporal dimension, as is the case with the ABL database, it is
necessary to create a dummy time by repeating the single spatial matrix a
number of times to enlarge data dimensionality, which is necessary to properly
address the next steps of the algorithm.

.

Figure 3: Sketch of matrix organization for one variable.

By performing the steps described above, a reduced and reshaped database
$X^{DS}$ is obtained. The data in this database are collected in the form of a
tensor:

$X^{DS}=X^{DS}_{ijkl}$ (4)

for $i=1,...,N_{t}$, $j=1,...,N_{sy}$, $k=1,...,\hat{N_{sx}}$,
$l=1,...,N_{var}$. Where $\hat{N_{sx}}=N_{sx}$ in the case of ABL and Cyl 2D
databases and $\hat{N_{sx}}=N_{sx}\cdot N_{z}$ in the case Cyl 3D database. 1
and $N_{sx}$ and $N_{sy}$ are the numbers of points selected in the
downsampling for $x$ and $y$, respectively, which varies for the different
test carried out, as it will be specified in the next section.

In the same way, the original tensor of the databases ($X$) is created and
then compared with the results of the neural network:

$X=X_{imnl}$ (5)

for $i=1,...,N_{t}$, $m=1,...,N_{y}$, $n=1,...,\hat{N_{x}}$,
$l=1,...,N_{var}$. Where $\hat{N_{x}}=N_{x}$ for ABL and Cyl 2D databases
$\hat{N_{x}}=N_{x}\cdot N_{z}$ for Cyl 3D database. The values of $N_{t}$,
$N_{y}$, $N_{x}$ and $N_{var}$ are indicated in Tab. 1.

$X^{DS}$ will be the input to the methodology described in this paper.
Finally, a summary of what has been explained in this section can be seen in
the sketch of Fig. 4

.

Figure 4: Sketch of the database organization before applying the methodology.
The dimensions for each case are indicated in Tab. 1.

##  4 Methodology

This section will describe the methodology followed. The dimension of a
three-, four- or five-dimensional databases has been strongly reduced to a few
points, mimicking data collected from experimental sensors. For simplicity,
these are called as the reduced databases or reduced tensors ($X^{DS}$). The
methodology begins by applying the singular value decomposition method (SVD)
[24] to the reduced tensor. Subsequently, the result of this decomposition is
introduced into the neural network to obtain the reconstructed tensor defined
as:

$\hat{X}=\hat{X}_{imnl}$ (6)

for $i=1,...,N_{t}$, $m=1,...,N_{y}$, $n=1,...,\hat{N_{x}}$,
$l=1,...,N_{var}$.

Note that the reconstructed tensor $\hat{X}$ has the same dimensions than the
original one previously described (Eq.5).

The sketch represented in Fig. 6 shows the general summary of the methodology
used.

Figure 5: Sketch of the methodology followed.

It should be remembered that the motivation of this study is to obtain an
algorithm that allows to increase the number of points in a database which
only few points available. Thus, the reduced tensor is the representation of a
real database with few points. The methodology can be summarized in two main
steps, each one corresponding to a part of this section: (i) SVD and (ii)
neural network (NN) architecture. In addition, the performance of the model
using a pre-processing techniques has been also evaluated.

Pre-processing techniques

Before applying SVD, a scaling method is used to pre-process the database,
homogenising the range value between all the input variables, avoiding in this
way to loose relevant information of the variables with smaller magnitude. Two
studies are carried out, scaling and without scaling the database, to test the
effect of this pre-processing technique in the final results. Although we will
see that scaling the database does not affect the architecture of the network.

The scaling method used in the present work ranges each variable between $0$
and $1$, as

$X^{SC}=X^{SC}_{l}=\frac{X^{DS}_{l}-min(X^{DS}_{l})}{max(X^{DS}_{l})-min(X^{DS}_{l})}$
(7)

for $l=1,...,N_{var}$. Where $min(X^{DS}_{l})$ and $max(X^{DS}_{l})$ are the
minimum and maximum values of the variable $X^{DS}_{l}$.

Accuracy and computational cost

On one hand, to perform the analysis of the results, two types of
characteristic errors will be taken into account: the Mean Square Error (MSE)
and the Relative Root Mean Square Error (RRMSE). The MSE is calculated as:

$MSE=\frac{X-\hat{X}}{N}$ (8)

while the RRMSE is computed as

$RRMSE=\frac{||X-\hat{X}||}{||X||},$ (9)

where $X$ is the original database, $\hat{X}$ is the reconstructed database
and $N$ is the total number of samples.

On the other hand, the computational cost associated to the model applied is
calculated as:

$t_{c}=t_{1}-t_{0}$ (10)

Figure 6: Sketch of the computational cost measurement.

where $t_{1}$ is the final time from the beginning of the code execution to
the end of the test part and $t_{0}$ is the initial time from the beginning of
the code execution to the end of the data loading.

###  4.1 Singular Value Decomposition: SVD

Once the reduced (and scaled) tensors are available, the next step in this
methodology is to apply a matrix singular value decomposition known as SVD
(Single Value Decomposition). This method was developed for fluid dynamics
applications by Sirovich [24] and consists of a matrix factorization that
captures the dominant directions of a database, where the vectors can shrink
or grow, depending on the magnitude of the singular values. The application of
this method is widely extended to various fields of application since it
allows to reduce the data dimensionality in complex dynamical systems. The
potential of this technique lies in being a method based on simple linear
algebra and being easily interpretable. In fluid dynamics, modes identify the
main patterns describing the flow physics. Hence, the model presented in this
paper considers the SVD modes, so it is based on physical principles.

SVD decomposes a matrix $\Pi$ of dimension $n\times m$ as the product of three
matrices:

$\stackunder{\mathrm{\Pi}}{\scriptstyle n\times m}=\stackunder{U}{\scriptstyle
n\times n}\ \stackunder{\Sigma}{\scriptstyle n\times m}\
\stackunder{V^{T}}{\scriptstyle m\times m}$ (11)

where $()^{T}$ denotes the matrix transpose and the matrix $\Sigma$ is a
diagonal matrix that contains the singular values of the matrix X ordered from
greater to smaller magnitude, being all of them greater or equal than 0.

It is possible to reduce the dimensionality of the data set by compressing the
matrix $\Pi$ without losing information, retaining only a few SVD modes, which
are associated the largest singular values (modelling the system dynamics).

SVD is applied to each snapshot $i$ of each variable $l$ in such a way that
the equation is redefined as follows:

$X^{DS}_{il}={U}^{il}\ \Sigma^{il}\ (V_{il})^{T}$ (12)

$U$, $\Sigma$ and $V$ matrices resulting from each snapshot $i$ are regrouped
in tensor form as follows:

$X_{U}^{DS}=U_{il}$ (13) $X_{S}^{DS}=\Sigma_{il}$ (14) $X_{V}^{DS}=V_{il}$
(15)

for $i=1,...,N_{t}$ and $l=1,...,N_{var}$.

The implementation in Python of SVD technique has been done by means of the
library numpy.linalg.svd. This library allows to obtain the $\Sigma$ matrix
already reduced with dimensions of $n\times k$ saving computational storage.
The $\Sigma$ matrix of dimensions $n\times k$ will be called matrix S. In
addition, the $U$ and $V$ matrices would have dimensions of $n\times k$ and
$m\times k$ respectively.

###  4.2 Neural Network architecture

Once the SVD method has been applied to the reduced tensors of each database,
the next step is to proceed to the reconstruction in order to recover the
original dimensions and compare the reconstructed data with the original ones
and obtain a percentage error. Different network architectures have been
tested, although only the one which provides the best results is described
below.

The designed network consists of 3 inputs corresponding to the 3 tensors
obtained by the SVD decomposition named $X^{DS}_{U}$, $X^{DS}_{S}$ and
$X^{DS}_{V}$, and an output corresponding to the tensor $\hat{X}$ which
represents the reconstructed tensor, which has the same dimensions as the
original one. Both tensors $X^{DS}_{U}$ and $X^{DS}_{V}$ enter an independent
autoencoder and on the last layer they are joined, with the tensor
$X^{DS}_{S}$, to form the output tensor. Each autoencoder consists of 5
layers. A sketch of the explained network architecture can be seen in Fig. 7
and a summary of the layer types is presented in Tab. 2. In the table there
some real variables: $bs$ (batch size), $N_{sy}$ and $\hat{N_{sx}}$ (number of
points selected in y-axis and x-axis, respectively), $N_{modes}$ (number of
SVD modes) and $N_{var}$ (number of variables).

Figure 7:  Sketch of the Neural Network architecture.  Table 2: Summary of the
layers that make up the network used where $bs$ is the batch size; $N_{sy}$
and $\hat{N}_{sx}$ are the number of points selected in y-axis and x-axis,
respectively; $N_{modes}$ is the number of SVD modes; $N_{var}$ is the number
of variables; $N_{y}$ and $N_{x}$ are the number of points in y- and x- axis
in the original databases.

Layer | Layer type | Neurons | Activation function | Dimensions
---|---|---|---|---
0 |  $X^{DS}_{U}$ (Input) |  |  | $(bs,\ N_{sy},\ N_{modes},\ N_{var})$
0 |  $X^{DS}_{V}$ (Input) |  |  | $(bs,\ N_{modes},\ \hat{N_{sx}},\ N_{var})$
0 |  $X^{DS}_{S}$ (Input) |  |  | $(bs,\ N_{modes},\ N_{modes},\ N_{var})$
1 | u 1 (Flatten) |  |  | $(bs,\ N_{sy}\times N_{modes}\times n_{var})$
1 | v 1 (Flatten) |  |  | $(bs,\ N_{modes}\times\hat{N_{sx}}\times N_{var})$
2 | u 2 (Dense) | 10 | ReLU | $(bs,\ 10)$
2 | v 2 (Dense) | 10 | ReLU | $(bs,\ 10)$
3 | u 3 (Dense) | $Ny\times N_{modes}\times N_{var}$ | Linear | $(bs,\ N_{y}\times N_{modes}\times N_{var})$
3 | v 3 (Dense) | $N_{modes}\times N_{x}\times N_{var}$ | Linear | $(bs,\ N_{modes}\times\hat{N_{x}}\times N_{var})$
4 |  $X^{US}_{U}$ (Reshape) |  |  | $(bs,\ N_{y},\ N_{modes},\ N_{var})$
4 |  $X^{US}_{V}$ (Reshape) |  |  | $(bs,\ N_{modes},\ \hat{N_{x}},\ N_{var})$
5 |  $\hat{X}_{imnl}$ (Out) |  |  | $(bs,\ N_{y},\ \hat{N_{x}},\ N_{var})$

In any neural network it is necessary to define a series of hyperparameters
that define the behavior of the network (Tab. 3). These hyperparameters are
the batch size, the loss function and the optimizer.

Table 3: Value of the hyperparameters relevant to different databases.

Hyperparameter | ABL | Cyl 2D | Cyl 3D
---|---|---|---
Batch size (bs) | 3 | 8 | 16
Loss function | MSE | MSE | MSE (Test 1) | RRMSE (Test 2)
Optimizer | Adam (0.001)

To clarify how the dimensions change along the methodology for different
downsamplings $K$ (to set the sensors), Fig. 8 has been included. Depending on
which database the methodology is applied to, the dimensions of the tensors
involved will change. In Tab. 4, the dimensions of these tensors have been
included for each of the cases included in the results section. It is
necessary to mention that in the case of ABL database the dimension of value
100 corresponds to the number of dummy snapshots ($(N_{t})_{dummy}$) and in
the case of Cyl 3D database, the dimension corresponding to $N_{x}$ and
$N_{sx}$ is equivalent to multiplying these by the dimension $N_{z}$.

Figure 8: Methodology sketch step by step with dimensions.  Table 4: Summary of the dimensions of the cases show in the results section. Database | k | Tensor | Tensor dimensions
---|---|---|---
ABL | 50 | $X^{DS}$ | ($N_{t}$, $N_{sy}$, $\hat{N}_{sx}$, $N_{var}$) | (100, 11, 41, 3)
$X^{DS}_{U}$ | ($N_{t}$, $N_{sy}$, $N_{modes}$, $N_{var}$) | (100, 11, 11, 3)
$X^{DS}_{S}$ | ($N_{t}$, $N_{modes}$, $N_{modes}$, $N_{var}$) | (100, 11, 11, 3)
$X^{DS}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{sx}$, $N_{var}$) | (100, 41, 11, 3)
$X^{US}_{U}$ | ($N_{t}$, $N_{y}$, $N_{modes}$, $N_{var}$) | (100, 501, 11, 3)
$X^{US}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{x}$, $N_{var}$) | (100, 11, 2001, 3)
$\hat{X}$ | ($N_{t}$, $N_{y}$, $\hat{N}_{x}$, $N_{var}$) | (100, 501, 2001, 3)
Below: 1:20 Above: 1:50 | $X^{DS}$ | ($N_{t}$, $N_{sy}$, $\hat{N}_{sx}$, $N_{var}$) | (100, 19, 41, 3)
$X^{DS}_{U}$ | ($N_{t}$, $N_{sy}$, $N_{modes}$, $N_{var}$) | (100, 19, 19, 3)
$X^{DS}_{S}$ | ($N_{t}$, $N_{modes}$, $N_{modes}$, $N_{var}$) | (100, 19, 19, 3)
$X^{DS}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{sx}$, $N_{var}$) | (100, 41, 19, 3)
$X^{US}_{U}$ | ($N_{t}$, $N_{y}$, $N_{modes}$, $N_{var}$) | (100, 19, 501, 3)
$X^{US}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{x}$, $N_{var}$) | (100, 2001, 19, 3)
$\hat{X}$ | ($N_{t}$, $N_{y}$, $\hat{N}_{x}$, $N_{var}$) | (100, 501, 2001, 3)
Cyl2D | 50 | $X^{DS}$ | ($N_{t}$, $N_{sy}$, $\hat{N}_{sx}$, $N_{var}$) | (151, 9, 4, 3)
$X^{DS}_{U}$ | ($N_{t}$, $N_{sy}$, $N_{modes}$, $N_{var}$) | (151, 9, 4, 3)
$X^{DS}_{S}$ | ($N_{t}$, $N_{modes}$, $N_{modes}$, $N_{var}$) | (151, 4, 4, 3)
$X^{DS}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{sx}$, $N_{var}$) | (151, 4, 4, 3)
$X^{US}_{U}$ | ($N_{t}$, $N_{y}$, $N_{modes}$, $N_{var}$) | (151, 449, 4, 3)
$X^{US}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{x}$, $N_{var}$) | (151, 4, 199, 3)
$\hat{X}$ | ($N_{t}$, $N_{y}$, $\hat{N}_{x}$, $N_{var}$) | (151, 449, 199, 3)
Cyl3D | 20 | $X^{DS}$ | ($N_{t}$, $N_{sy}$, $\hat{N}_{sx}$, $N_{var}$) | (299, 5, 2x64, 3)
$X^{DS}_{U}$ | ($N_{t}$, $N_{sy}$, $N_{modes}$, $N_{var}$) | (299, 5, 5, 3)
$X^{DS}_{S}$ | ($N_{t}$, $N_{modes}$, $N_{modes}$, $N_{var}$) | (299, 5, 5, 3)
$X^{DS}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{sx}$, $N_{var}$) | (299, 5, 2x64, 3)
$X^{US}_{U}$ | ($N_{t}$, $N_{y}$, $N_{modes}$, $N_{var}$) | (299, 100, 5, 3)
$X^{US}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{x}$, $N_{var}$) | (299, 5, 40x64, 3)
$\hat{X}$ | ($N_{t}$, $N_{y}$, $\hat{N}_{x}$, $N_{var}$) | (299, 100, 40x64, 3)
15 | $X^{DS}$ | ($N_{t}$, $N_{sy}$, $\hat{N}_{sx}$, $N_{var}$) | (299, 5, 2x64, 3)
$X^{DS}_{U}$ | ($N_{t}$, $N_{sy}$, $N_{modes}$, $N_{var}$) | (299, 5, 5, 3)
$X^{DS}_{S}$ | ($N_{t}$, $N_{modes}$, $N_{modes}$, $N_{var}$) | (299, 5, 5, 3)
$X^{DS}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{sx}$, $N_{var}$) | (299, 5, 2x64, 3)
$X^{US}_{U}$ | ($N_{t}$, $N_{y}$, $N_{modes}$, $N_{var}$) | (299, 100, 5, 3)
$X^{US}_{V}$ | ($N_{t}$, $N_{modes}$, $\hat{N}_{x}$, $N_{var}$) | (299, 5, 40x64, 3)
$\hat{X}$ | ($N_{t}$, $N_{y}$, $\hat{N}_{x}$, $N_{var}$) | (299, 100, 40x64, 3)

It is important to understand that the network consists of two autoencoders
that work in parallel and meet at the output layer with the input
$X^{DS}_{S}$, which only appears at this last part of the neural network.
Therefore, the reconstruction error used to improve the weights is calculated
by comparing the output tensor $\hat{X}$ with the original tensor, i.e., it is
not calculated by comparing the tensors $X^{US}_{U}$ and $X^{US}_{V}$
separately with the original ones.

The strategy followed to find out if the network is able to predict in time
consists of the following. The total number of available snapshots has been
divided into three groups: training, validation and test. The snapshots
corresponding to the training group will be used to train the network, the
snapshots from the validation set will be used to validate the network results
into the training part and the snapshots in the test group will be used to
test the network after being trained. The distribution of snapshots into the
three groups is different to each database. Values for $T_{train}$, $T_{val}$
and $T_{test}$ are giving in Tab. 5.

Figure 9: Snapshot organization sketch for time prediction.  Table 5: Number of snapshots for training, validation and test sets for each one of the databases analysed. Database | $T_{train}$ | $T_{val}$ | $T_{test}$ | T
---|---|---|---|---
ABL | 72 | 13 | 15 | 100
Cyl 2D | 68 | 12 | 71 | 151
Cyl 3D | 170 | 30 | 99 | 299

The open source platform known as TensorFlow has been used to design the
network. Specifically, the Keras programming interface included in TensorFlow
has been used.

##  5 Results and Discussions

This section will show in detail the performance of the methodology explained
in Sec. 4 applied to each one of the different databases.

###  5.1 Atmospheric Boundary Layer

When applying the method with neural networks to the ABL database, the results
have been separated into two different tests. On test 1, the downsampling has
been performed equispaced, while on test 2, the downsampling has been non-
homogeneous, choosing more points on the area near the wall, located at low
$y$ values. In addition, the scaling of the data has also been tested. Once
the SVD is applied, an additional dimension for the time is created but the
same snapshot is repeat all the time, to enlarge data dimensionality. Then,
the three input tensors are then separated into the training, validation and
test sets. Afterwards, the ones selected for the training part are introduced
into the network and after an iterative process the best weights that define
this problem are obtained. In the test, the reduced flow field is
reconstructed. The number of snapshots selected for the training, validation
and test datasets are indicated in Tab. 5.

Test 1 corresponds to the results of the reconstruction when the sparse data
are obtained homogeneously. Tab.6 shows the values of the MSE according to
different downsampling. It can be seen that the best results are obtained for
a downsampling of 1:50 and that increasing the number of points on the reduced
tensor does not improve the error suggesting that overfitting is ocurring in
this case. This is because the neural network architecture is developed to
work on extreme conditions (very sparse data), representing realistic and
industrial problems. However, it can be seen that for a downsampling of 1:90
the same error in pressure and temperature is achieved as for a downsampling
of 1:50. So, there is an optimal performance for this model.

Table 6: MSE error obtained in different homogeneous downsampling for the ABL
database.

Downsampling | Streamwise velocity | Pressure | Temperature | TOTAL
---|---|---|---|---
1:10 |  No scaling | 0.012 | 0.052 | 0.000006 | 0.021
|  Scaling | 0.00031 | 0.30 | 0.000001 | 0.10
1:50 |  No scaling | 0.00033 | 0.052 | 0.000006 | 0.017
|  Scaling | 0.00034 | 0.30 | 0.000001 | 0.10
1:90 |  No scaling | 0.0019 | 0.052 | 0.000006 | 0.018
|  Scaling | 0.012 | 0.30 | 0.000005 | 0.10

In Fig. 10 the results obtained applying a downsampling of 1:50 are shown. On
the third column we present the results without scaling the data and on the
fourth column, the results by scaling the data. Comparing the representation
of the reconstructed tensor with the original tensor, there are hardly any
visual differences. It is also observed that the loss of information that
occurs in the velocity variable in the reduced tensor (blue band), is
recovered in the reconstructed tensor (the blue band disappears). The same
happens with the yellow stripe in the temperature variable.

Figure 10: Results of the ABL base neural network method performing 1:50
homogeneous downsampling. The horizontal axis indicates the distance in meters
from the study field [0,5000] and the vertical axis the height in meters
[0,500].

A zoomed-in view of the area near the wall (from 0 to 15 meters) can be seen
in 11. The downsampling representation of the velocity indicates that the
velocity is zero in this region. This occurs because the points chosen for
downsampling correspond as the zero velocity points in the original
representation. Nevertheless, this method is capable to reconstruct this
region with high accuracy.

Figure 11: Same as figure 10 but the vertical axis dimensions are [0,15]
meters, near the wall.

Test 2 shows the results of performing a non-homogeneous downsampling to the
original tensor of the ABL database. More points are taken in the lower zone
of the boundary layer, where the streamwise velocity shows a greater contrast.

The results presented correspond to a downsampling where in the $x$ dimension
1 point out of 50 points has been chosen but in the $y$ dimension it has been
divided into two equal parts: in the first part from 0 m to 250 m 1 point out
of 20 points has been chosen and in the second part from 250 m to 500 m 1
point out of 50 points has been chosen. The result provides a reduced tensor
of reduced dimensions $3\times 19\times 41$ where there are 41 points in $x$
and 19 points in $y$. In Fig. 12, the selected points are displayed
graphically.

Figure 12: Sketch of the non-homogeneous downsampling performed on the ABL
database.

The results obtained from the neural network with non-homogeneous downsampling
are shown in Fig. 13. The differences with the test 1 are barely noticeable on
most of the domain. The only appreciable difference is in the reduced
representations of velocity and temperature, second column of Fig. 13. Due to
the fact that more points have been chosen in the lower zone, the zero
velocity band (blue band) and the yellow band in temperature have been
reduced.

Figure 13: Results of the neural network method on the basis of ABL performing
non-homogeneous downsampling. The horizontal axis indicates the distance in
meters from the study field [0,5000] and the vertical axis the height in
meters [0,500].

Finally, Tab.7 and 8 summarizes and compares the results obtained for the ABL
database comparing the RRMSE and MSE in all tests performed, respectively. In
Tab.7 the computational cost is also included.

Table 7: Percentage of RRMSE error committed when constructing the ABL
database and computational cost ($t_{c}$). Test 1 refers to homogeneous
downsampling results (1:50) and Test 2 refers to non-homogeneous downsampling
results (Below: 1:20; Above: 1:50).

Method | Streamwise velocity | Pressure | Temperature | TOTAL |  $t_{c}$(s)
---|---|---|---|---|---
NN Test 1 | No scaling | 0.12 | 0.00023 | 0.00081 | 0.00023 | 93.9
Scaling | 0.12 | 0.00039 | 0.00056 | 0.00056 | 80.1
NN Test 2 | No scaling | 0.73 | 0.00023 | 0.00081 | 0.00026 | 99.3
Scaling | 0.11 | 0.00095 | 0.00044 | 0.00095 | 99.1

Table 8: MSE error made when constructing the ABL database. Test 1 refers to
homogeneous downsampling results (1:50) and Test 2 refers to non-homogeneous
downsampling results (Below: 1:20; Above: 1:50).

Method | Streamwise velocity | Pressure | Temperature | TOTAL
---|---|---|---|---
NN Test 1 | No scaling | 0.00034 | 0.052 | 0.000006 | 0.017
Scaling | 0.0003 | 0.30 | 0.000005 | 0.10
NN Test 2 | No scaling | 0.012 | 0.052 | 0.000006 | 0.021
Scaling | 0.0003 | 0.88 | 0.000002 | 0.29

Regarding the need of scaling the data, the results show that there is no
significant improvement in terms of errors either. However, scaling the data
reduces the computational cost because the data are less dispersed. Therefore,
the choice of whether or not to scale the data will depend on which objective
is more important: computational cost or to achieve a lower error in certain
variables. Finally, a comparison of test 1 with test 2 leads to the conclusion
that the non-homogeneous downsampling applied improves the results very
slightly but at the computational cost increases.

###  5.2 Two-dimensional flow past a circular cylinder

The methodology for this new database is the same as the one applied to the
ABL database, although in this case the flow is unsteady. As a reminder, the
network architecture is exactly the same as indicated in Sec. 4 showing the
robustness and the generalization of the model presented.

For this database, different results are presented varying the number of
points chosen for downsampling. Also, tests have been performed scaling the
input data. A summary of the MSE obtained in the different tests is shown in
Tab. 9. The table shows the representation of a representative snapshot of the
reduced matrix for the U velocity component. The original tensor of this
database has dimensions of $3\times 449\times 199\times 151$ so performing a
1:50 downsampling gives a reduced tensor of $3\times 9\times 4\times 151$ and
a downsampling of 1:10 one tensor of $3\times 45\times 20\times 151$.

The table shows that for the case without scaling, the total error decreases
as more points are introduced into the network. However, this trend is not
observed when the data is scaled since for a downsampling of 1:10 the error
shoot up. This suggest that overfitting is ocurring: too much data is
introduced into the network. This is related to both the number of data and
the dispersion of those data, so it occurs when the data are scaled and not
when they are not scaled.

Table 9: MSE error obtained in different downsampling for the two-dimensional
cylinder database.

Downsampling | Velocity U | Velocity V | Vorticity | TOTAL
---|---|---|---|---
1:10 |  No scaling | 0.011 | 0.0023 | 0.033 | 0.016
|  Scaling | 0.019 | 0.010 | 4.26 | 1.43
1:20 |  No scaling | 0.011 | 0.0024 | 0.063 | 0.025
|  Scaling | 0.0021 | 0.0017 | 0.40 | 0.14
1:30 |  No scaling | 0.0037 | 0.00073 | 0.10 | 0.035
|  Scaling | 0.00086 | 0.00069 | 0.16 | 0.054
1:40 |  No scaling | 0.0059 | 0.0019 | 0.19 | 0.068
|  Scaling | 0.0022 | 0.0015 | 0.22 | 0.076
1:50 |  No scaling | 0.0016 | 0.0057 | 0.31 | 0.11
|  Scaling | 0.0050 | 0.0034 | 0.36 | 0.12

To show that the error for the 1:10 case with scaling is triggered due to
overfitting, the number of modes of the SVD decomposition that will enter the
network has been reduced. The comparison of the MSE obtained for a
downsampling of 1:10 with and without a mode reduction is shown in Tab. 10. As
seen, reducing the number of modes, i.e., reducing the amount of data
introduced on the neural network, improves the performance of the
interpolation.

Table 10: MSE error committed for a downsampling of 1:10 by reducing the
number of modes to 10 in the tow-dimensional cylinder database.

Downsampling | Velocity U | Velocity V | Vorticity | TOTAL
---|---|---|---|---
1:10 |  Mode reduction | 0.0035 | 0.0042 | 0.71 | 0.24
|  No mode reduction | 0.019 | 0.010 | 4.26 | 1.43

Finally, graphical representations of the results for the most extreme
downsampling (1:50) are shown in Fig. 14. The input to the network are
represented on the second column of Fig. 14, while the reconstructions are
represented on the third column for non scaling data and on the fouth column
for scaling data. As seen, some differences can be appreciated between the
reconstructions and the original snapshots (first column). The reconstruction
of points has been carried out on the three variables that make up this
database ($v_{x}$, $v_{y}$ and vorticity), however, it has been decided to
analyze the two most important physical variables ($v_{x}$ and $v_{y}$)
because with the composition of these variables the vorticity can be obtained.

Figure 14: Results of the neural network method of two-dimensional cylinder
base data by performing 1:50 downsampling. The horizontal coordinate axis
indicates the horizontal distance between [-1,8] and the vertical axis
indicates the vertical distance between [-2,2]. Snapshot number = 80.

The results are then compared using error analysis. Tab. 11 shows the results
of the MSE for a downsampling of 1:50. Due to use the MSE as loss function, an
attempt is made to optimize this error. If a comparison is made between the
results with scaling and without it, the scaling results provides better
results.

The results corresponding to the RRMSE are shown in Tab. 12, as well as an
average of the computational cost of each algorithm. These errors are higher
than the MSE seen above because they are triggered by making errors when
reconstructing data whose value is 0, although it is still interesting to
analyze as it provides the percentage of data that could not be reconstructed.
In the same way than before, the scaling results are around 5% order better
than no scaling ones.

Finally, 5% and 10% of random noise has been added to the original database to
test the response of the NN architecture in more realistic situations (as in
the case of experimental databases). These results show that the error is
higher than the tests without noise but remains within similar values as in
the previous cases, but increasing the error with a level similar to the noise
range.

In terms of computational cost, the cost of interpolations is lower than that
of NNs, but once the model, which is robust, is developed, the computational
cost of NNs is still low (less than 1 min) and provides better results than
the simple interpolation.

Table 11: MSE error committed when reconstructing the two-dimensional cylinder
database with 1:50 downsampling.

Method | Velocity U | Velocity V | TOTAL
---|---|---|---
NN | No scaling | 0.016 | 0.0057 | 0.011
Scaling | 0.0050 | 0.0034 | 0.0042
NN noise test | 5% interference | 0.025 | 0.0015 | 0.013
10% interference | 0.030 | 0.018 | 0.024

Table 12: Percentage of RRMSE error committed when reconstructing the two-
dimensional cylinder database with 1:50 downsampling and computational cost.

Method | Velocity U | Velocity V | TOTAL |  $t_{c}$ (s)
---|---|---|---|---
NN | No scaling | 14.1 | 29.0 | 21.5 | 54.4
Scaling | 7.9 | 22.4 | 15.1 | 55.4
NN noise test | 5% interference | 17.2 | 46.9 | 32.5 | 20.4
10% interference | 18.1 | 51.0 | 34.5 | 22.6

###  5.3 Three-dimensional flow past a circular cylinder

The last database to be tested is the three-dimensional flow past a circular
cylinder.

Two main tests are going to be presented. The difference between these two
test is the change in the loss function. On test 1, the MSE (eq. (8)) of the
validation set is the loss function, while on test 2, the RRMSE (eq. (9)) is
going to be tried.

The network methodology followed for this database is practically identical to
the methodology followed with the rest of the databases. The main difference
is that the database has $5$ dimensions (variables, time and three space
dimensions), while the other cases have just $4$. Therefore, if the same
network architecture is to be used, it has been necessary to add an extra step
before applying the SVD decomposition. This extra step consists of a reshaping
of the $5$-dimensional tensor to a $4$-dimensional tensor. The chosen option
has been to join the dimension corresponding to the points in $x$ with the
points in $z$: from an original tensor of dimension $3\times 100\times
40\times 64\times 299$ an extreme downsampling of 1:20 has been performed on
the $x$ and $y$ dimensions, obtaining a reduced tensor of dimension $3\times
5\times 2\times 64\times 299$ and subsequently applying the
redimensionalization, it remains a tensor of $3\times 5\times 128\times 299$.
To this last tensor, the SVD decomposition is applied and the resulting
matrices are those that are introduced in the network. The output of the
network will be a tensor of $3\times 100\times 2560\times 299$ which will have
to be resized again to separate the $x$ and $z$ dimensions.

As done previously, a study has been carried out with different downsampling
in order to know which amount of data provides the best results. Tab. 13 shows
the MSE obtained for each downsampling. The maximum downsampling done in the
database has been 1:20, as the original database has only $40$ and $100$
points in the streamwise and normal directions, respectively. With this
downsampling the points are reduced to 2 and 5. As seen in Tab. 13, the total
errors are of the same order of magnitude except for those belonging to the
1:10 downsampling with scaling, where the error increases one order of
magnitude due to overfitting. The best results are observed for 1:15
downsampling without data scaling, however, the results obtained for 1:20
downsampling are very good considering the amount of data used. Scaling the
data does not seem to improve the performance of the neural network.

Table 13: MSE error obtained in different downsampling for the three-
dimensional cylinder database.

Downsampling | Velocity U | Velocity V | Velocity W | TOTAL
---|---|---|---|---
1:10 | Test 1 | No scaling | 0.012 | 0.0090 | 0.00043 | 0.0071
| Scaling | 0.028 | 0.025 | 0.00094 | 0.018
| Test 2 | No scaling | 0.011 | 0.0099 | 0.00031 | 0.0070
| Scaling | 0.022 | 0.019 | 0.00064 | 0.014
1:15 | Test 1 | No scaling | 0.0064 | 0.0046 | 0.00042 | 0.0038
| Scaling | 0.0092 | 0.0090 | 0.00054 | 0.0062
| Test 2 | No scaling | 0.0051 | 0.0039 | 0.00035 | 0.0031
| Scaling | 0.0079 | 0.0065 | 0.00044 | 0.0049
1:20 | Test 1 | No scaling | 0.011 | 0.0091 | 0.00050 | 0.0070
| Scaling | 0.0091 | 0.0087 | 0.00052 | 0.0061
| Test 2 | No scaling | 0.0095 | 0.0079 | 0.00046 | 0.0059
| Scaling | 0.0082 | 0.010 | 0.00047 | 0.0062

Some graphical results from the reconstruction of this database are shown in
Figs. 15 and 16. Figure 15 shows the results for a representative snapshot of
test 1. The reduced representations (second column) bear little resemblance to
the original representations. However, the reconstructed representations
(third column) are really close to the original ones providing very good
results considering that 5 points in $x$ and 2 points in $y$ have been
introduced. However, In the case of the spanwise velocity (third row), the
reconstruction graph is more different than the others compared with the
original database. This is due to the fact that most of the $v_{z}$ velocity
terms are zero-valued as can be seen in the original representation. Then,
taking into account the above, the reconstruction of these points is
associated with a very high relative error.

The results corresponding to test 2 are shown in Fig. 16 for the same snapshot
and the same z-plane as the graphs presented for test 1. Comparing the results
of both tests, no relevant differences can be seen. In the following section,
a comparison of errors between the two tests will be made in order to know the
differences. Regarding the scaled results that appear in the fourth column of
both figures, it is observed that the maximum $U$ velocity points that appear
in the original representation (yellow areas) are reconstructed but the values
reached are not the correct ones and therefore, the MSE observed in the
downsampling table is higher in the cases with scaling.

Figure 15: Results of the neural network method of three-dimensional cylinder
base data by performing 1:20 downsampling. The horizontal coordinate axis
indicates the horizontal distance between [0,10] and the vertical axis
indicates the vertical distance between [-2,2]. Snapshot number = 200. z=0.
Figure 16: Same as for figure 15 but for test 2.

Fig. 17 includes the results for the optimal downsampling according to the
error table discussed above. Increasing the points in $x$ and $y$ from 2 and 5
(downsampling 1:20) to 3 and 7 (downsampling 1:15), the performance of the
neural network is highly improved, particularly for the streamwise velocity
$U$, where negligible differences can be seen compared to the original
snapshot.

Figure 17: Results of the neural network method of 3D Cylinder base data by
performing 1:15 downsampling. The horizontal coordinate axis indicates the
horizontal distance between [0,10] and the vertical axis indicates the
vertical distance between [-2,2]. Snapshot number = 200. z=0.

Tab. 14 and Tab. 15) shows the MSE and RRMSE reconstruction errors,
respectively, for a downsampling of 1:20.

From the MSE Tab. 14 it is observed that the lowest absolute error is obtained
for the case of test 2 (loss function RRMSE) without scaling the data.
However, the lowest RRMSE is given for the case of test 1 with the data
scaled, according to Tab. 15. In addition, it is observed that the RRMSE for
the different cases are quite close. In the case of the spanwise velocity, the
RRMSE errors are higher than in the rest of the variables due to the fact that
most of the $v_{z}$ velocity terms are zero and that makes the reconstructed
relative error rises. Nevertheless, the MSE error is not as high as the RRMSE
error.

The computational cost is higher than for the 2D Cylinder database, as it is
more complex. Nevertheless, it is good as the values hardly exceed 5 minutes.

Table 14: MSE error committed when reconstructing the 3D Cylinder database
with a downsampling of 1:20.

Method | Velocity U | Velocity V | Velocity W | TOTAL
---|---|---|---|---
Neural Network Test 1 | Without scaling | 0.011 | 0.0091 | 0.00050 | 0.0070
With scaling | 0.0091 | 0.0087 | 0.00052 | 0.0061
Neural Network Test 2 | Without scaling | 0.0095 | 0.0079 | 0.00046 | 0.0059
With scaling | 0.0082 | 0.010 | 0.00047 | 0.0063

Table 15: Percentage of RRMSE error committed when reconstructing the 3D
Cylinder database with 1:20 downsampling and computational cost.

Method | Velocity U | Velocity V | Velocity W | TOTAL | Cost (s)
---|---|---|---|---|---
Neural Network Test 1 | Without scaling | 11.3 | 28.5 | 89.3 | 14.4 | 169.5
With scaling | 10.1 | 27.9 | 90.7 | 12.3 | 300.8
Neural Network Test 2 | Without scaling | 10.3 | 26.4 | 85.6 | 13.3 | 123.8
With scaling | 9.5 | 29.9 | 86.3 | 13.6 | 196.6

##  6 Conclusions

The present work explores the posibility of using hybrid models, combining
physical principles with strategies of machine learning, to reconstruct fluid
dynamics databases from sensor measurements. For this purpose, the data
dimensionality is reduced using a singular value decomposition (SVD), which
extract the main physical patterns modeling the flow. From the reduced
database, a very simple and efficient neural network based on autoencoders has
been designed to perform to reconstruct a database from a very reduced number
of points, which represent sensor measurements. The methodology, which is
completely new and simple, has been tested on different databases to check
also the robustness of the model developed. More specifically, the method is
applied in three different databases representing a turbulent atmospheric
boundary layer (ABL) database and the two- and three- dimensional flow past a
circular cylinder.

The performance of the method has been tested using different donwsamplings of
the original database, representing different sensor positions, and also the
effect of scaling the original database have been studied. The results show
that method presented is not only capable to reconstruct the original flow
field, as in the case of the ABL, but also is capable to predict the temporal
evolution of new reconstructed databases (also coming from sensor
measurements), with errors smaller than $5-10$%, depending on the type of
downsampling and complexity of the case. The model has also been proved to
perfectly work with noisy databases and also when the sensors are not
collected equidistant in space, but grouped into some specific areas of the
field.

The results obtained shows that the novel method, which is based on a very
simple neural network architecture (minimizing computational costs and
increasing generalization capabilities) is robust and supports all types of
databases tested.

It has been shown that the neural network, in addition to reconstructing, is
also able to predict in time. This, together with the low computational cost
that has been obtained, is a very valuable tool that could save a lot of data
simulation costs.

Finally, summary tables with the RRMSE and MSE errors presented throughout the
results analysis have been included in order to have an overview of the error
order achieved with this neural network. The errors presented in the tables
present the same order than the errors from simulations found in the
literature of similar problems [25] [26] [27] [28]. The present work achieves
errors of the same order of magnitude in much less time, demonstrating its
suitability for the reconstruction of fluid dynamics databases.

To sum up, this work proposes a hybrid physics-based machine learning model
with a simple, robust and generalizable architecture, which allows
reconstructing databases from very few points mimicking a sensors network and
with a low computational cost.

Table 16: Summary of the RRMSE error rate in reconstructing the different
databases and computational cost.

Database |  Method | Variable 1 | Variable 2 | Variable 3 | TOTAL | Cost (s)
---|---|---|---|---|---|---
ABL  1:50 | NN Test 1 | No scaling | 0.12 | 0.00023 | 0.00081 | 0.00023 | 93.9
Scaling | 0.12 | 0.00039 | 0.00056 | 0.00056 | 80.1
NN Test 2 | No scaling | 0.73 | 0.00023 | 0.00081 | 0.00026 | 99.3
Scaling | 0.11 | 0.00095 | 0.00044 | 0.00095 | 99.1
Cylinder 2D  1:50 | NN | No scaling | 14.1 | 29.0 |  | 21.5 | 54.4
Scaling | 7.9 | 22.4 |  | 15.1 | 55.4
NN (noise test) | 5% interference | 17.2 | 46.9 |  | 20.4 | 20.4
10% interference | 18.1 | 51.0 |  | 34.5 | 22.6
Cylinder 3D  1:20 | NN Test 1 | No scaling | 11.3 | 28.5 | 89.3 | 14.4 | 169.5
Scaling | 10.1 | 27.9 | 90.7 | 12.3 | 300.8
NN Test 2 | No scaling | 10.3 | 26.4 | 85.6 | 13.3 | 123.8
Scaling | 9.5 | 29.9 | 86.3 | 13.6 | 196.6

Table 17: Summary of the MSE error made when reconstructing the different
databases.

Database |  Method | Variable 1 | Variable 2 | Variable 3 | TOTAL
---|---|---|---|---|---
ABL  1:50 | NN Test 1 | No scaling | 0.00034 | 0.052 | 0.000006 | 0.017
Scaling | 0.0003 | 0.30 | 0.0000005 | 0.10
NN Test 2 | No scaling | 0.012 | 0.052 | 0.000006 | 0.021
Scaling | 0.0003 | 0.88 | 0.000002 | 0.29
Cylinder 2D  1:50 | NN | No scaling | 0.016 | 0.0057 |  | 0.011
Scaling | 0.0050 | 0.0034 |  | 0.0042
NN (noise test) | 5% interference | 0.025 | 0.0015 |  | 0.013
10% interference | 0.030 | 0.018 |  | 0.024
Cylinder 3D  1:20 | NN Test 1 | No scaling | 0.011 | 0.0091 | 0.00050 | 0.0070
Scaling | 0.0091 | 0.0087 | 0.00052 | 0.0061
NN Test 2 | No scaling | 0.0095 | 0.0079 | 0.00046 | 0.0059
Scaling | 0.0082 | 0.010 | 0.00047 | 0.0063

## Acknowledgements

A.C. and S.L.C. acknowledge the grant PID2020-114173RB-I00 funded by MCIN/AEI/
10.13039/501100011033. P.D, A.C. and S.L.C. acknowledge the support of
Comunidad de Madrid through the call Research Grants for Young Investigators
from Universidad Politécnica de Madrid. A.C. also acknowledges the support of
Universidad Politécnica de Madrid, under the program ‘Programa Propio’.

## References

  * [1] M. Zyskowski, Aircraft simulation techniques used in low-cost, commercial software, in: AIAA Modeling and Simulation Technologies Conference and Exhibit, 2003, p. 5818.
  * [2] D. Ciliberti, P. Della Vecchia, F. Nicolosi, A. De Marco, Aircraft directional stability and vertical tail design: A review of semi-empirical methods, Progress in Aerospace Sciences 95 (2017) 140–172.
  * [3] R. M. Hall, S. H. Woodson, J. R. Chambers, Overview of the abrupt wing stall program, Progress in Aerospace Sciences 40 (7) (2004) 417–452.
  * [4] S. Heinz, A review of hybrid rans-les methods for turbulent flows: Concepts and applications, Progress in Aerospace Sciences 114 (2020) 100597.
  * [5] K. Veggeberg, High channel-count aircraft noise mapping applications, Sound and Vibration 43 (5) (2009) 14.
  * [6] S. Liu, Numerical simulation of hypersonic aerodynamics and the computational needs for the design of an aerospace plane, Tech. rep., RAND CORP SANTA MONICA CA (1992).
  * [7] W. Wong, N. Qin, N. Sellars, H. Holden, H. Babinsky, A combined experimental and numerical study of flow structures over three-dimensional shock control bumps, Aerospace Science and Technology 12 (6) (2008) 436–447.
  * [8] S. L. Brunton, B. R. Noack, P. Koumoutsakos, Machine learning for fluid mechanics, Annual review of fluid mechanics 52 (2020) 477–508.
  * [9] A. Güemes, C. S. Vila, S. Discetti, Super-resolution gans of randomly-seeded fields, arXiv preprint arXiv:2202.11701.
  * [10] H. Kim, J. Kim, S. Won, C. Lee, Unsupervised deep learning for super-resolution reconstruction of turbulence, Journal of Fluid Mechanics 910.
  * [11] T. Bolton, L. Zanna, Applications of deep learning to ocean data inference and subgrid parameterization, Journal of Advances in Modeling Earth Systems 11 (1) (2019) 376–399.
  * [12] K. Fukami, K. Fukagata, K. Taira, Super-resolution reconstruction of turbulent flows with machine learning, Journal of Fluid Mechanics 870 (2019) 106–120.
  * [13] N. B. Erichson, L. Mathelin, Z. Yao, S. L. Brunton, M. W. Mahoney, J. N. Kutz, Shallow neural networks for fluid flow reconstruction with limited sensors, Proceedings of the Royal Society A 476 (2238) (2020) 20200097.
  * [14] Y. Bengio, A. Courville, P. Vincent, Representation learning: A review and new perspectives, IEEE transactions on pattern analysis and machine intelligence 35 (8) (2013) 1798–1828.
  * [15] N. C. Thompson, K. Greenewald, K. Lee, G. F. Manso, The computational limits of deep learning, arXiv preprint arXiv:2007.05558.
  * [16] J. R. Garratt, The atmospheric boundary layer, Earth-Science Reviews 37 (1-2) (1994) 89–134.
  * [17] J. Cuxart, C. Yagüe, G. Morales, E. Terradellas, J. Orbe, J. Calvo, A. Fernández, M. Soler, C. Infante, P. Buenestado, et al., Stable atmospheric boundary-layer experiment in spain (sables 98): a report, Boundary-layer meteorology 96 (3) (2000) 337–370.
  * [18] D. Hargreaves, N. G. Wright, On the use of the k–$\epsilon$ model in commercial cfd software to model the neutral atmospheric boundary layer, Journal of wind engineering and industrial aerodynamics 95 (5) (2007) 355–369.
  * [19] C. Jackson, A finite-element study of the onset of vortex shedding in flow past variously shaped bodies, Journal of fluid Mechanics 182 (1987) 23–45.
  * [20] D. Barkley, R. D. Henderson, Three-dimensional floquet stability analysis of the wake of a circular cylinder, Journal of Fluid Mechanics 322 (1996) 215–241.
  * [21] J. M. Vega, S. Le Clainche, Higher order dynamic mode decomposition and its applications, Academic Press, 2020.
  * [22] S. Le Clainche, J. M. Pérez, J. M. Vega, Spatio-temporal flow structures in the three-dimensional wake of a circular cylinder, Fluid Dynamics Research 50\.
  * [23] S. Le Clainche, J. M. Pérez, J. M. Vega, Spatio-temporal flow structures in the three-dimensional wake of a circular cylinder, Fluid Dynamics Research 50 (5) (2018) 051406.
  * [24] L. Sirovich, Turbulence and the dynamics of coherent structures. iii. dynamics and scaling, Quarterly of Applied mathematics 45 (3) (1987) 583–590.
  * [25] R. Abadía-Heredia, M. López-Martín, B. Carro, J. I. Arribas, J. M. Pérez, S. Le Clainche, A predictive hybrid reduced order model based on proper orthogonal decomposition combined with deep learning architectures, Expert Systems with Applications 187 (2022) 115910.
  * [26] S. Le Clainche, J. M. Vega, Higher order dynamic mode decomposition to identify and extrapolate flow patterns, Physics of Fluids 29 (8) (2017) 084102.
  * [27] J. Kou, S. Le Clainche, W. Zhang, A reduced-order model for compressible flows with buffeting condition using higher order dynamic mode decomposition with a mode selection criterion, Physics of Fluids 30 (1) (2018) 016103.
  * [28] S. Le Clainche, E. Ferrer, A reduced order model to predict transient flows around straight bladed vertical axis wind turbines, Energies 11 (3) (2018) 566\.