Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000. xxxx

All authors are with the BioMedical Data analysis group
(https://www.uis.no/en/bmdlab). This study was supported and approved by the
Regional ethic committee project 2012/1499.

Corresponding author: Luca Tomasetti (e-mail: luca.tomasetti@uis.no).

# CT Perfusion is All We Need: 4D CNN Segmentation of Penumbra and Core in
Patients With Suspected Ischemic Stroke

Luca Tomasetti1  Kjersti Engan1  Liv Jorunn Høllesli1,2  Kathinka Dæhli
Kurz1,2  and Mahdieh Khanmohammadi1  Department of Electrical Engineering and
Computer Science, University of Stavanger, Stavanger, 4021, Norway  Stavanger
Medical Imaging Laboratory (SMIL), Department of Radiology, Stavanger
University Hospital, Stavanger, 4068, Norway

###### Abstract

Precise and fast prediction methods for ischemic areas comprised of dead
tissue, core, and salvageable tissue, penumbra, in acute ischemic stroke (AIS)
patients are of significant clinical interest. They play an essential role in
improving diagnosis and treatment planning. Computed Tomography (CT) scan is
one of the primary modalities for early assessment in patients with suspected
AIS. CT Perfusion (CTP) is often used as a primary assessment to determine
stroke location, severity, and volume of ischemic lesions. Current automatic
segmentation methods for CTP mostly use already processed 3D parametric maps
conventionally used for clinical interpretation by radiologists as input.
Alternatively, the raw CTP data is used on a slice-by-slice basis as 2D+time
input, where the spatial information over the volume is ignored. In addition,
these methods are only interested in segmenting core regions, while predicting
penumbra can be essential for treatment planning. This paper investigates
different methods to utilize the entire 4D CTP as input to fully exploit the
spatio-temporal information, leading us to propose a novel 4D convolution
layer. Our comprehensive experiments on a local dataset of 152 patients
divided into three groups show that our proposed models generate more precise
results than other methods explored. Adopting the proposed _4D mJ-Net_ , a
Dice Coefficient of 0.53 and 0.23 is achieved for segmenting penumbra and core
areas, respectively. The code is available on GitHub.

###### Index Terms:

4D Convolution, Acute Ischemic Stroke, Computed Tomography Perfusion, Deep
Neural Network, Image Segmentation.

=-21pt

##  I Introduction

An acute ischemic stroke (AIS) generally occurs if a segment of the supplying
arteries of the brain is occluded by a blood clot and prevents the regular
flow of oxygen-rich blood to the capillaries in the brain tissue. The ischemic
area can roughly be divided into two different types: 1) _penumbra_ , areas
where the tissue is still vital but critically hypoperfused [1]; and 2) _core_
, referring to non-salvageable tissue. If blood flow is not restored timely,
penumbra regions may develop rapidly into irreversibly damaged core regions.
Therefore, a fast and accurate understanding of ischemic areas to plan the
treatment and tailor further procedures to every single patient is
fundamental.

The recommended modalities for diagnostic imaging in AIS patients are Computed
Tomography (CT) and Magnetic Resonance Imaging (MRI) [2]. In the initial
stages of an acute ischemic stroke, CT Perfusion (CTP) has proven to be a fast
and beneficial tool for evaluating both diagnosis and prognosis [3]. MRI with
Diffusion-weighted imaging (DWI) or non-contrast CT (NCCT) are commonly
utilized, after treatment, to assess the _final infarct areas_ (FIAs) [4].
These imaging modalities are obtained hours or days after the patient’s
treatment.

CTP is a four-dimensional (4D) spatio-temporal examination to assess the
passage of blood in the brain. It is performed by acquiring a series of three-
dimensional (3D) CT scans of a specific portion of the brain at time intervals
during contrast agent injection. By using an iodinated contrast agent, density
changes in the brain tissue over time can be analyzed. The shape and height of
the time density curve depend on the brain tissue’s perfusion [5]. The raw 4D
CTP contains vast data; hence, detecting ischemic strokes can be challenging
for neuroradiologists, primarily due to the need for precise diagnosis and for
rapid treatment decisions.

To overcome this challenge, medical doctors rely on a set of clinically
interpretable parametric maps (PMs) generated from the 4D CTP scan [5, 6]. The
generated PMs reduce the temporal dimension and produce 3D volumes. Commonly
used PMs are cerebral blood flow (CBF), cerebral blood volume (CBV), time-to-
maximum (TMAX), and time-to-peak (TTP). CBF represents the blood supply in the
brain at a given time; CBV refers to the blood volume present at a given time
in a brain region; TMAX is the flow-scaled residue function in the tissue;
while TTP shows the time until the contrast agent reaches the tissue [5].
Maximum intensity projection (MIP) is also usually generated. MIP images are
calculated as the maximum Hounsfield unit (HU) value over the time sequence of
the CTP, providing a 3D volume from the 4D acquisition of CTP. Although PMs
provide helpful information about ischemic brain tissue, extracting them from
the 4D CTP scans limits the spatio-temporal information only to specific
subsets of information [7]. Several methods [8, 4, 9, 10, 11] have used
thresholding techniques to predict the ischemic areas from the PMs. However,
simple thresholding approaches over-simplify the complexity in AIS [12, 13].

In the past years, Deep Neural Networks (DNNs), and especially Convolutional
Neural Networks (CNNs), have been successfully applied in numerous medical
applications: image classification tasks [14, 15, 16, 17, 18], automatic video
analysis [19, 20], and activity recognition [21, 22, 23]. Automatic image
segmentation adopting U-Net structure [24] and its numerous variants have
produced innovative outcomes for several applications [25, 26, 27, 28, 29].

Several DNNs have been proposed for AIS applications to predict and segment
only the FIAs using CT studies in combination with PMs derived from CTP scans
as input [30, 31, 32, 33, 34]. Other researchers have proposed architectures
to segment the ischemic lesion (i.e., the core) from the images obtained at
hospital admission. [35] were the first to implement a CNN with a set of PMs
as input for ischemic core segmentation. [36] proposed a few-shot self-
supervised architecture for hypoperfused (core + penumbra) tissue segmentation
using a combination of PMs and raw scans as input of the model. The work
demonstrated the feasibility of using self-supervised techniques for the
segmentation of this type of tissue. [37] introduced a machine learning
segmentation method to delineate hypoperfused tissue, demonstrating the
capabilities of this methodology over classic thresholding approaches. They
used four PMs as input features for their model. However, a general problem
with all the methods mentioned above is relying on commercial CTP software and
using heavily pre-processed information (i.e., PMs) rather than taking
advantage of the totality of the raw 4D CTP scans.

DNNs are more suitable for discovering information from raw data.
Nevertheless, relying on raw data (directly exploiting the temporal and
spatial dimensions) is scarcely explored in the literature for AIS
applications. The task is challenging because of the low contrast and low
signal-to-noise ratio in the CTP scans. Relatively few studies proposed DNN
models with encouraging results, exploiting the temporal dimension to assess
acute stroke lesions using 4D CTP scans. [7] utilized CTP images to create 2D
matrices in which each row is a voxel, and each column is a time point. The 2D
matrices are used as input for a model that shows encouraging results in
differentiating healthy tissue from FIAs. [38] promoted a 2D+time symmetry-
aware CNN-based architecture to segment FIAs using solely CTP scans. Their
work estimated the irreversibly damaged areas, demonstrating the possibilities
of using 4D CTP images for this task. [39] used a U-Net-like structure for
segmenting FIAs using CTP scans as an input plus contra-lateral information.
Results were promising, but further research is needed due to their far-from-
ideal registration of the contra-lateral information. [40] introduced a two-
step model for estimating FIAs using the 4D CTP series as input. They first
generate an arterial input function and later deconvolve it with a singular
value decomposition approach to find the infarction. [41] designed a framework
based on Temporal Convolution Network to predict AIS FIAs from 4D CTP studies.
Due to memory constraints, they independently processed each 2D slice of the
4D CTP dataset. In their recent work, [42] also proposed an extension of their
model where 3D+time tensors of the ipsilateral stroke hemisphere are used as
input to predict FIAs. [43] proposed a DNN that predicts the FIAs directly
from 4D raw CTP plus patients metadata. Their proposed architecture relied on
a series of 3D Convolution layers; the input is a list of 4D CTP scans sampled
at different resolutions. Their method presented promising segmentation
results; however, their main target was to estimate final infarct volume
allowing clinicians to simulate different treatments and gain insight for the
procedures. They were not taking into consideration the penumbra in their
study. Plus, the quality of the ground truth images is debatable since they
rely on NCCT follow-up images acquired between 24 hours and five days after
patient’s admission. It has been reported that FIAs can grow after 24 hours in
NCCT measurements [44].

All of the segmentation methods mentioned above rely on ground truth labels
obtained from DWI and/or NCCT hours or days after the patient’s admission
since they are predicting FIAs. Even though follow-up images (DWI and NCCT)
represent the gold standard for estimating core [4], there are some
limitations with these techniques [45, 46]. Follow-up images can only be used
to assess FIAs but not penumbra regions. Plus, some studies have demonstrated
that the detected FIAs can be partially reverse in DWI performed in an early
time window [46, 47, 48].

The studies of [7, 38, 39, 43, 40, 41, 42] indicate the potential of 4D data
in AIS prediction. However, they all consider FIAs and an appropriate method
is still required to simultaneously handle the spatio-temporal information for
segmenting the ischemic _core_ and _penumbra_ regions. Understanding the
penumbra’s extension during the ischemic stroke’s first stages is crucial for
treatment decision [49, 50]. To the best of our knowledge, our work in
Tomasetti et al. [50] using machine learning, and later in Tomasetti et al.
[51, 52] using DNN, were the first and only to segment both core and penumbra
areas. In [50, 52] the PMs were used as input, in [51] 2D + time CTP images
were segmented slice-by-slice.

In this paper, we present and investigate three novel models to segment the
two ischemic regions (core and penumbra), where the input is the entire 4D CTP
scans arranged in different ways to exploit the spatio-temporal nature of the
data. We compare all models with previous work based on PMs [52] and slice-by-
slice CTP [51], and two methods proposed by [41, 42].

The main contributions of this work can be summarized in four points:

  1. 1.

We develop a novel 4D convolution layer that processes the 4D raw data.

  2. 2.

We implement the 4D convolution layer and propose a DNN model, _4D mJ-Net_ ,
to segment ischemic core and penumbra areas from 4D CTP scans.

  3. 3.

We extend previous methods [41, 51] to be used directly with 4D CTP scans as
input.

  4. 4.

To assess the results, we use manual annotations obtained by two expert
neuroradiologists from the 4D CTP data upon patients’ admission. We also
demonstrate the feasibility of our proposed methods by comparing their
performances with existing models that rely on different inputs.

##  II Data Material

A section of the brain is repeatedly scanned during the passage of 40 ml
iodine-containing contrast agent (Omnipaque 350 mg/ml) and 40 ml isotonic
saline in a cubital vein with a flow rate of 6 ml/s to highlights changes in
the tissue; the scan delay was four seconds. Each brain slice contains a fixed
number of time points $t_{\text{max}}$ representing the temporal dimension.
The width and height of each image are $512\times 512$ pixels with a
resolution of 0.4258 mm/pixel and a slice thickness of 5mm. The first twenty
time points are acquired every 1s, while the remaining ten images are every
2s.

CTP scans from 152 patients collected between January 2014 and August 2020
formed the dataset. 137 of these patients had an AIS with a visible perfusion
deficit. During the diagnostic workup, the remaining 15 patients were admitted
with suspected strokes but were determined not to have suffered from a stroke
episode after the diagnostic workup. Raw perfusion data from the CTP
examination was used to generate PMs with the software “syngo.via” from
Siemens Healthineers, with manufacturer default settings. The arterial input
function was automatically selected, with few exceptions where it was chosen
manually (e.g., severe cardiac failure).

The patients were divided into the following groups: 77 patients with large
vessel occlusion (LVO), 60 patients with non-large vessel occlusion (Non-LVO),
and the remaining 15 patients without ischemic stroke (WIS). Based on CT
angiography, LVO was defined as occlusion of any of the following arteries:
the internal carotid artery, M1 and proximal M2 segment of the middle cerebral
artery, A1 segment of the anterior cerebral artery, P1 segment of the
posterior cerebral artery, basilar artery, and vertebral artery occlusion.
Non-LVO was defined as patients with perfusion deficit with more distal artery
occlusion or with perfusion deficit without visible artery occlusion.

TABLE I: Division in training, validation, and test dataset.

| Training (#; %) | Validation (#; %) | Test (#; %) | Tot. (#; %)
---|---|---|---|---
LVO | 42; 54.5 | 16; 20.8 | 19; 24.7 | 77; 50.6
Non-LVO | 36; 60 | 13; 21.7 | 11; 18.3 | 60; 30.5
WIS | 9; 60 | 3; 20 | 3; 20 | 15; 9.8
Total | 87; 58.6 | 32; 19.7 | 33; 21.7 | 152; 100

The dataset is randomly split into a training, validation, and test set. The
percentage of the three subsets (LVO, Non-LVO, WIS) is equally distributed
among the sets, as shown in Table I.

###  II-A Ground truth

The manual annotations are based on the entire CT dataset, including the PMs
derived from CTP. MRI performed during the first days after hospital admission
was also utilized. Two expert neuroradiologists manually annotated ground
truth images by utilizing the complete set of the CT examination (NCCT, CT
angiography, and CTP), which includes PMs from the CTP (CBV, CBF, TTP, TMAX)
and the MIP images. The PMs were visually assessed. In general, ischemic
regions with increased TTP and TMAX and reduced CBF but preserved CBV were
considered as penumbra, while areas with additionally reduced CBV were deemed
as core. Additionally, the MRI examination, including DWI, was conducted
within 1 to 3 days after the CT examination, and clinical information was used
to assist in generating the ground truth images. The annotations were
performed using an in-house developed software in Matlab111The code is
publicly available at https://github.com/Biomedical-Data-Analysis-
Laboratory/CTP-Matlab.

##  III Background theory

###  III-A Notation

Table II presents the various formal notations adopted in the remainder of the
paper. Let the data obtained from a CTP scan be defined as a 4D tensor
$V\in\mathbb{R}^{(X\times Y\times Z\times T)}$. After a series of pre-
processing steps (details in Sec. III-B), we define the 4D tensor as
$\widetilde{V}\in\mathbb{R}^{(X\times Y\times Z\times T)}$. The four
dimensions of a CTP scan are defined as width ($X$), height ($Y$), depth
($Z$), and time ($T$). The list of time points in the time dimension is given
by $t=[t_{j}|\forall j\in\\{1,2,\cdots,t_{\text{max}}\\}]$, where
$t_{\text{max}}$ is the last time point of the list. We indicate how the
notation superscript adopts the time dimension in the various inputs.
Furthermore, we define $z=[z_{i}|\forall i\in\\{1,2,\cdots,z_{\text{max}}\\}]$
as the list of brain slices in the depth dimension, where $z_{\text{max}}$
corresponds to the last slice. We illustrate how the depth dimension is being
used in the inputs through the notation subscript. Fig. 3 displays the input
combination of all the techniques.

TABLE II: List of formal notations used in the paper.

Notation | Description
---|---
$t=[t_{j}|\forall j\in\\{1,2,\cdots,t_{\text{max}}\\}]$ | List of all the time points.
$t_{\text{max}}$ |  Last time point in the time dimension.
$z=[z_{i}|\forall i\in\\{1,2,\cdots,z_{\text{max}}\\}]$ | List of all the slices.
$z_{\text{max}}$ |  Last slice in the depth dimension.
$\mathcal{I}=\\{i-1,i,i+1\\}$ |  Set of indexes $i$, plus its neighbours $i-1$ and $i+1$.
$z_{I}=\\{z_{i-1},z_{i},z_{i+1}\\}$ |  Set of slices: $z_{i-1},z_{i},z_{i+1}$.
$V\in\mathbb{R}^{(X\times Y\times Z\times T)}$ | 4D raw CTP input.
$\mathcal{C}=\\{$healthy brain, penumbra, core$\\}$ | Set of classes.
$I^{t_{j}}_{z_{i}}\in\mathbb{R}^{(X\times Y)}$ |  2D brain slice $z_{i}$ at time point $t_{j}$.
$P_{z_{i}}\in\mathbb{R}^{(X\times Y)}$ |  2D probability output of brain slice $z_{i}$.
$\widetilde{\cdot}$ |  Input after pre-processing steps.
$\bar{\cdot}$ |  List of inputs
$\varphi(\cdot)$ | Concatenation function.
$\widehat{\cdot}$ |  Concatenated inputs after passing through $\varphi(\cdot)$.
$\bar{V}^{t}_{z_{i}}=[\widetilde{I}_{z_{i}}^{t_{j}}|\forall t_{j}\in t]\in\mathbb{R}^{(X\times Y)}$ |  List of 2D images $\widetilde{I}_{z_{i}}$ for all the time points $t$. Input for _2D-TCN_ (Sec. IV-A3).
$\widehat{V}_{z_{i}}^{t}=\varphi(\widetilde{I}_{z_{i}}^{t_{j}}|\forall t_{j}\in t)\in\mathbb{R}^{(X\times Y\times T)}$ |  2D+time volume of slice $z_{i}$. Input for _mJ-Net_ (Sec. IV-A2).
$\widehat{V}^{t_{j}}_{z_{\mathcal{I}}}=\varphi(\widetilde{V}^{t_{j}}_{z_{i-1}},\widetilde{V}^{t_{j}}_{z_{i}},\widetilde{V}^{t_{j}}_{z_{i+1}})\in\mathbb{R}^{(X\times Y\times Z)}$ |  3D volume of slices $z_{\mathcal{I}}$ for a time point $t_{j}$.
$\bar{V}^{t}_{z_{\mathcal{I}}}=[\widehat{V}^{t_{j}}_{z_{\mathcal{I}}}|\forall t_{j}\in t]\in\mathbb{R}^{(X\times Y\times Z)}$ |  List of 3D volumes of slices $z_{\mathcal{I}}$ for all the time points $t$. Input for _3D-TCN_ (Sec. IV-B1).
$\bar{V}_{z_{\mathcal{I}}}^{t}=[\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t}]\in\mathbb{R}^{(X\times Y\times Z\times T)}$ |  List of 2D+time volumes for slices $z_{\mathcal{I}}$. Input for _3D+time mJ-Net_ (Sec. IV-B2).
$\widehat{V}^{t}_{z_{\mathcal{I}}}=\varphi(\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t})\in\mathbb{R}^{(X\times Y\times Z\times T)}$ |  4D Tensor of slices $z_{\mathcal{I}}$ over all the time points $t$. Input for _4D mJ-Net_ (Sec. IV-B3).

All methods return a 3D output, segmenting the images $P_{z_{i}}$ slice-by-
slice. The segmented 2D image $P_{z_{i}}$ corresponds to a brain slice $z_{i}$
at index $i$. The predicted image $P_{z_{i}}$ contains brain tissue segmented
with the classes $\mathcal{C}$ (if any): healthy brain, penumbra, and core.

###  III-B Pre-processing steps

The 4D CTP dataset underwent a series of pre-processing steps to extract brain
tissue from the raw CTP scans. The raw CTP studies are saved as DICOM files.
The pre-processing steps can be summarized as follow:

  1. 1.

Co-registration of all the images in the 4D CTP scan using the first time
point image as the frame of reference in order to correct possible motion
artifacts. An intensity-based image registration with similarity
transformation was used in this step.

  2. 2.

All the registered CTP scans were encoded into HU values to have a known
quantitative scale to describe radiodensity efficiently. To calculate the HU
value for a voxel $V$ with a rescale slope (RS) and a rescale intercept (RI)
extracted from the DICOM header:
$V(x,y,z,t)_{\text{HU}}=V(x,y,z,t)\cdot\text{RS}+\text{RI}$

  3. 3.

Brain extraction of CT studies plays an essential role in stroke imaging
research [53, 54]. An automatic brain extraction method designed by [54] was
selected for this purpose due to its proven efficiency with CT datasets and
public availability.

  4. 4.

Gamma correction ($\gamma=0.5$) and histogram equalization were also performed
after step 3.

  5. 5.

Finally, z-score ($z$) on the enhanced 4D tensor is applied to normally
distribute the data.

The input for all the methods (except for the _Multi-input PMs_ ’s approach)
follows the same pre-processing steps. These steps were performed to improve
the quality of the images by enhancing the contrast. An additional resampling
step for all the images was performed to ensure uniform distribution in
temporal dimension. The effects of the pre-processing steps and re-sampling
are examined in an ablation study in Sec. V-C.

###  III-C Convolution in many dimensions

A handful of DNN methods have been proposed to exploit 4D data with a full 4D
Convolution (4D-Conv) layer. [55] and [56] proposed a 4D spatio-temporal
convolutional network to optical coherence tomography force estimation. They
demonstrated that using the full 4D data information yields better
performances than 3D data. [57] introduced a 4D CNN to segment cardiac
volumetric sequences using CT scans, showing the advantages of using their
proposed architecture compared to a classic 3D CNN.

In the remainder of this manuscript, let define
$I(x,y,z,t)\in\mathbb{R}^{4},\mathcal{H}(w,h,d,p)\in\mathbb{R}^{4}$ as a 4D
tensor and a 4D kernel, respectively. The $x$ and $w$ indicate the width of
the 4D structures; $y$ and $h$ express the height dimension; $z$ and $d$
define the depth dimension, while $t$ and $p$ represent the time dimension of
the 4D structures. Like a 3D Convolution (3D-Conv) can be represented as the
sum of multiple 2D Convolution (2D-Conv) along the depth dimension, a 4D-Conv
operation can be described as the sum of multiple 3D-Conv along the temporal
dimension. The loop rearrangement to avoid repeated 3D-Conv operations allows
a true (non-separable) 4D convolution operation [57].

Let define a 2D-Conv $g^{\prime\prime}(x,y)$, where a 2D input image
$I(x,y)\in\mathbb{R}^{2}$ is convolved with a 2D kernel
$\mathcal{H}(w,h)\in\mathbb{R}^{2}$ as:

$\displaystyle g^{\prime\prime}(x,y)$
$\displaystyle=\mathcal{H}(w,h)\circledast I(x,y)$
$\displaystyle=\sum_{i=0}^{w-1}\sum_{j=0}^{h-1}\mathcal{H}(i,j)I(x+\widetilde{w}-i,y+\widetilde{h}-j)$

where $\circledast$ is the convolution operation.
$\widetilde{h}\equiv\lfloor\frac{1}{2}(h-1)\rfloor,\widetilde{w}\equiv\lfloor\frac{1}{2}(w-1)\rfloor$
correspond to the half-width, and half-height of the kernel $\mathcal{H}$.

Thereafter, let us define a 2D-Conv $g^{\prime\prime}(x,y,z)$ with a 2D kernel
$\mathcal{H}(w,h)\in\mathbb{R}^{2}$ and a 3D input
$I(x,y,z)\in\mathbb{R}^{3}$. Since the convolution operation is performed
slice by slice over the third dimension, the 3D input can be seen as a list of
2D input $I(x,y,z)=\\{I(x,y,z_{m})|\forall
m\in\\{1,\dots,z_{\text{max}}\\}\\}$, where $I(x,y,z_{m})\in\mathbb{R}^{2}$
are the coordinates $(x,y)$ at slice $z_{m}$:

$\displaystyle g^{\prime\prime}(x,y,z)$
$\displaystyle=\mathcal{H}(w,h)\circledast I(x,y,z)$ $\displaystyle
g^{\prime\prime}(x,y,z_{m})$
$\displaystyle=\sum_{i=0}^{w-1}\sum_{j=0}^{h-1}\mathcal{H}(i,j)I(\tilde{x},\tilde{y},z_{m})\;\forall
m\in\\{1,\dots,z_{\text{max}}\\}$

where $\tilde{x}\equiv x+\widetilde{w}-i$ and $\tilde{y}=y+\widetilde{h}-j$.
Fig. 1 presents a visual representation of the 2D-Conv
$g^{\prime\prime}(x,y,z)$ with a 3D input and a 2D kernel.

(a) 2D-Conv with a 3D input and a 2D kernel

(b) 3D-Conv with a 3D input and a 3D kernel

Figure 1: Visual examples of (a) 2D-Conv and (b) 3D-Conv. Both examples
utilize a 3D input, and no padding is applied. Each box corresponds to a pixel
value.

Furthermore, let define a 3D-Conv operation $g^{\prime\prime\prime}(x,y,z)$ of
a 3D kernel $\mathcal{H}(w,h,d)\in\mathbb{R}^{3}$ and a 3D input volume
$I(x,y,z)\in\mathbb{R}^{3}$ as:

$\displaystyle g^{\prime\prime\prime}(x,y,z)=\mathcal{H}(w,h,d)\circledast
I(x,y,z)$
$\displaystyle=\sum_{k=0}^{d-1}\sum_{i=0}^{w-1}\sum_{j=0}^{h-1}\mathcal{H}(i,j,k)I(x+\widetilde{w}-i,y+\widetilde{h}-j,z+\widetilde{d}-k)$

where $\widetilde{d}\equiv\lfloor\frac{1}{2}(d-1)\rfloor$ is the half-depth of
$\mathcal{H}$. Fig. 1 displays an example of a 3D-Conv with a 3D input and 3D
kernel. If we define the 3D input $I(x,y,z)$ as before, then the output of a
3D-Conv $g^{\prime\prime\prime}(x,y,z)$ can be rewritten as the sum of
multiple 2D-Conv operations over the third dimension:

$\displaystyle g^{\prime\prime\prime}(x,y,z)$
$\displaystyle=\mathcal{H}(w,h,d)\circledast I(x,y,z)$
$\displaystyle=\sum_{k=0}^{d-1}\mathcal{H}(w,h,k)I(x,y,z+\widetilde{d}-k)$
$\displaystyle=\sum_{k=0}^{d-1}g^{\prime\prime}(x,y,z+\widetilde{d}-k)$

where $\mathcal{H}(w,h,k)\in\mathbb{R}^{2}$ is a 2D kernel at index $k$, while
$I(x,y,z+\widetilde{d}-k)\in\mathbb{R}^{2}$ is a 2D image at slice
$z+\widetilde{d}-k$ where $[z+\widetilde{d}-k\in z_{m}|\forall
m\in\\{1,\dots,z_{\text{max}}\\}]$. The output dimension is defined as:
$\text{dim}(g^{\prime\prime\prime})=\text{dim}(I)-\text{dim}(\mathcal{H})+1$;
thus, if the $\text{dim}(I)\equiv\text{dim}(\mathcal{H})$, then
$\text{dim}(g^{\prime\prime\prime})=1$.

Moreover, lets define a 3D-Conv operation $g^{\prime\prime\prime}(x,y,z,t)$ of
a 3D kernel $\mathcal{H}(w,h,d)\in\mathbb{R}^{3}$ with a 4D input tensor
$I(x,y,z,t)\in\mathbb{R}^{4}$. The 4D tensor $I(x,y,z,t)$ can be seen as a
list of $I(x,y,z,t)=[I(x,y,z,t_{n})|\forall
n\in\\{1,\dots,t_{\text{max}}\\}]$, where each element in the list corresponds
to the coordinates $(x,y,z)$ of a $t_{n}$ element in the temporal dimension.
Then, the 3D-Conv operation $g^{\prime\prime\prime}(x,y,z,t)$ can be seen as:

$\displaystyle g^{\prime\prime\prime}(x,y,z,t)=\mathcal{H}(w,h,d)\circledast
I(x,y,z,t)$
$\displaystyle=\sum_{k=0}^{d-1}\sum_{i=0}^{h-1}\sum_{j=0}^{w-1}\mathcal{H}(i,j,k)I(\tilde{x},\tilde{y},\tilde{z},t_{n})\;\forall
n\in\\{1,\dots,t_{\text{max}}\\}$

where $\tilde{x}\equiv x+\widetilde{w}-i$, $\tilde{y}=y+\widetilde{h}-j$, and
$\tilde{z}\equiv z+\widetilde{d}-k$.

If we use a 2D+time kernel $\mathcal{H}(w,h,p)\in\mathbb{R}^{3}$ and we define
$I(x,y,z,t)$ as a list of $I(x,y,z,t)=[I(x,y,z_{m},t)|\forall
m\in\\{1,\dots,z_{\text{max}}\\}]$, where each element in the list corresponds
to the coordinates $(x,y,t)$ of a slice $z_{m}$, then the 3D-Conv operation
$g^{\prime\prime\prime}(x,y,z,t)$ can be rewritten as:

$\displaystyle g^{\prime\prime\prime}(x,y,z,t)=\mathcal{H}(w,h,p)\circledast
I(x,y,z,t)$ $\displaystyle
g^{\prime\prime\prime}(x,y,z_{m},t)=\mathcal{H}(w,h,p)\circledast
I(x,y,z_{m},t)\;\forall m\in\\{1,\dots,z_{\text{max}}\\}$
$\displaystyle=\sum_{l=0}^{p-1}\mathcal{H}(w,h,l)I(x,y,z_{m},t+\widetilde{p}-l)\;\forall
m\in\\{1,\dots,z_{\text{max}}\\}$

where $\widetilde{p}\equiv\lfloor\frac{1}{2}(p-1)\rfloor$ is the temporal
dimension of the kernel $\mathcal{H}$ halved.

A 4D-Conv $g^{\prime\prime\prime\prime}(x,y,z,t)$ of a 4D input
$I(x,y,z,t)\in\mathbb{R}^{4}$ and a 4D kernel
$\mathcal{H}(w,h,d,p)\in\mathbb{R}^{4}$ can be defined as:

$\displaystyle
g^{\prime\prime\prime\prime}(x,y,z,t)=\mathcal{H}(w,h,d,p)\circledast
I(x,y,z,t)$
$\displaystyle=\sum_{l=0}^{p-1}\sum_{k=0}^{d-1}\sum_{i=0}^{w-1}\sum_{j=0}^{h-1}\mathcal{H}(i,j,k,l)I(\tilde{x},\tilde{y},\tilde{z},\tilde{t})$

where $\tilde{x}\equiv x+\widetilde{w}-i$, $\tilde{y}=y+\widetilde{h}-j$,
$\tilde{z}\equiv z+\widetilde{d}-k$, and $\tilde{t}\equiv t+\widetilde{p}-l$.

Figure 2: Visual representation of a 4D-Conv layer. The input is
$\widehat{V}^{t}_{z_{\mathcal{I}}}=\varphi(\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t})\in\mathbb{R}^{(X\times
Y\times Z\times T)}$. A series of 3D-Conv operations are calculated over a 4D
input. Several groups $(G_{i-1},G_{i},G_{i+1})$ are generated, one for each
volume involved in the operation.

Finally, in a similar way, we define a 4D-Conv
$g^{\prime\prime\prime\prime}(x,y,z,t)$ and a 4D kernel $\mathcal{H}(w,h,d,p)$
as the sum of multiple 3D-Conv over a specific dimension, i.e., the third
dimension. The 4D kernel $\mathcal{H}(w,h,d,p)$ can be seen as a list of
$\mathcal{H}(w,h,k,p)|\forall k\in d-1$, where $\mathcal{H}(w,h,k,p)$ is a
2D+time volume of the $k$th slice over the entire $p$ elements in the temporal
dimension:

$\displaystyle g^{\prime\prime\prime\prime}(x,y,z,t)$
$\displaystyle=H(w,h,d,p)\circledast I(x,y,z,t)$
$\displaystyle=\sum_{k=0}^{d-1}\mathcal{H}(w,h,k,p)I(x,y,z+\widetilde{d}-k,t)$
$\displaystyle=\sum_{k=0}^{d-1}g^{\prime\prime\prime}(x,y,z+\widetilde{d}-k,t)$

where $I(x,y,z+\widetilde{d}-k,t)\in\mathbb{R}^{3}$ is a 2D+time volume at
slice $z+\widetilde{d}-k$ over the totality $t$ element in the temporal
dimension, and $[z+\widetilde{d}-k\in z_{m}|\forall
m\in\\{1,\dots,z_{\text{max}}\\}]$

Fig. 2 gives a visual explanation of the proposed 4D-Conv layer. The input for
our 4D-Conv layer is a 4D tensor $\widehat{V}^{t}_{z_{\mathcal{I}}}$: the
2D+time volume of the $i$th brain slice $\widehat{V}^{t}_{z_{i}}$ over all the
time points $t$, and the two 2D+time volumes of the neighboring brain slices
($\widehat{V}^{t}_{z_{i-1}}$ and $\widehat{V}^{t}_{z_{i+1}}$). The 4D-Conv
layer uses a loop rearrangement with three 3D-Conv groups
$(G_{i-1},G_{i},G_{i+1})$, one for each volume slice involved in the
operation. In each group $G_{i}$, several 3D-Conv layers are created. All
convolution layers in each group shared the weights. Each 3D-Conv layer is
used for a single input volume. The number of layers depends on the legal
subscript indexes, i.e., for the group $G_{i-1}$, there are two 3D-Conv layers
since the legal subscript indexes are $\\{i-1,i\\}$: the indexes are given by
the current volume slice $z_{i-1}$ and the only neighboring volume slice
$z_{i}$. The output of each group is a set of feature volumes summed together.
The resulting feature volumes are stacked together to keep the same dimension
as the input.

##  IV Existing methods & Proposed 4D methods

In this paper, we present three novel deep learning (DL) approaches that
accommodate 4D input data. In the remainder of the paper, they are called
_3D-TCN_ (Sec. IV-B1), _3D+time mJ-Net_ (Sec. IV-B2), and _4D mJ-Net_ (Sec.
IV-B3).

Together with the proposed approaches, we implemented and compared the
_2D-TCN_ [41] (Sec. IV-A3), the _3D-TCN-SE_ [42] (Sec. IV-A4), and the _mJ-
Net_ [51] (Sec. IV-A2) to validate the performances of our models. We also
compare the models with a method that uses a set of PMs as input [52]. In the
remainder of the paper, we call this architecture _Multi-input PMs_ (Sec.
IV-A1). Fig. 3 visually compares the input utilized for the various
approaches.

Figure 3: Visual comparison of the input for each implemented approach. Every
4D CTP patient’s study $V\in\mathbb{R}^{(X\times Y\times Z\times T)}$
undergoes a series of pre-processing steps to enhance each CTP scan. Approach
1 (_Multi-input PMs_) [52] accepts a list of PMs generated from a CTP study in
input. Approach 2 (_mJ-Net_) [51] use a 2D+time volume
$\widehat{V}_{z_{i}}^{t}(x,y)$ as input. Approach 3 (_2D-TCN_) and Approach 4
(_3D-TCN-SE_) follows the model proposed by [41] and [42], respectively. The
resulting 4D tensor is fed to one of the approaches. The proposed approach 5
(_3D-TCN_), approach 6 (_3D+time mJ-Net_), and approach 7 (_4D mJ-Net_) take
in input the entire 4D CTP processed data $\widetilde{V}$.

###  IV-A Existing methods

####  IV-A1 Approach 1: Multi-input PMs

The _Multi-input PMs_ model was proposed in [52]. This architecture was used
as a baseline study because all the PMs were input for the model. The input
for the architecture is a list of PMs for each brain slice $z_{i}$:
$\text{PMs}_{z_{i}}$, as shown in Table III. The loss function implemented is
the Focal Tversky loss (FTL) [58]; for a specific class $c$, the FTL is
defined as:

$\text{FTL}(x,y)=\sum_{c}^{\mathcal{C}}(1-\text{TI}_{c})^{1/\gamma}$

where $\gamma\geq 1$ is a hyper-parameter that forces the loss function to
focus more on less accurate predictions that have been misclassified [58].
Denoting $x_{i,c}\in[0,1]$ as the probability of the $i$th predicted pixel to
belong to class $c$; $y_{i,c}\in\\{0,1\\}$ as the pixel $i$ with class $c$ in
a ground truth image, $\text{TI}_{c}$ is the Tversky index (TI) for a class
$c$ defined as:

$\text{TI}_{c}=\frac{\sum_{i=1}^{M\times N}x_{i,c}y_{i,c}}{\sum_{i=1}^{M\times
N}x_{i,c}y_{i,c}+\alpha\sum_{i=1}^{M\times
N}\widehat{x_{i,c}}y_{i,c}+\beta\sum_{i=1}^{M\times
N}x_{i,c}\widehat{y_{i,c}}}$

where $\widehat{x_{i,c}}=1-x_{i,c}$ is the probability that the $i$th pixel is
not of class $c$, and $\widehat{y_{i,c}}=1-y_{i,c}$ represents the complement
of pixel $i$ in a ground truth image. The hyper-parameters $\alpha$ and
$\beta$ control the trade-off between precision and recall. We refer the
reader to [52] for a more extensive explanation and discussion about this
approach.

####  IV-A2 Approach 2: mJ-Net

The _mJ-Net_ approach was proposed in [51]. As presented in Table III, the
input $\widehat{V}_{z_{i}}^{t}(x,y)$ for _mJ-Net_ is a 2D+time volume of the
same brain slice $z_{i}$ at index $i$ over all the time points $t$. We define
the dimension of this input as 2D+time; the first dimension of the input is
time.

The loss function used for this method is the soft Dice Coefficient loss
(SDCL) [59]. The SDCL is a modified version of the Dice Coefficient score
mainly used in medical domains where the classes to predict are highly
unbalanced due to a small region of interest compared to the background of the
scans. The SDCL can be written as:

$\displaystyle\text{SDCL}(x,y)=\sum_{c}^{\mathcal{C}}\left(1-\frac{2\sum_{i}^{M\times
N}x_{i,c}y_{i,c}}{\sum_{i}^{M\times N}x_{i,c}^{2}+\sum_{i}^{M\times
N}y_{i,c}^{2}}\right)$

The first section of the model contains 3D-Conv layers to extract information
from the temporal dimension, while the second part follows the classic U-Net
structure [24]. For more details about the _mJ-Net_ approach, we refer the
reader to [51].

####  IV-A3 Approach 3: 2D Temporal Convolutional Network

For comparison reasons, we implemented the method proposed by [41]. We call
this architecture _2D-TCN_ in the remainder of the paper. Fig. 4 provides a
general architecture overview. They proposed a Temporal Convolutional Network
(TCN), which has been shown to outperform conventional neural networks in
different tasks [60]. Moreover, a TCN has a lower memory requirement for
training than other Recurrent Neural Networks [60].

(a) _2D-TCN_ architecture proposed by [41].

(b) _3D-TCN-SE_ proposed by [42]

Figure 4: Visual comparison between (a) the _2D-TCN_ architecture [41] and (b)
the _3D-TCN-SE_ [42].

The _2D-TCN_ was trained with the exact implementation as the original work
(see Table III for more details). The _2D-TCN_ model receives the 4D CTP scans
in input re-sampled to 1 second per time point. The 4D input is processed as a
list of 2D brain slices $z_{i}$ for each time point $t$. Thus, the actual
input for the _2D-TCN_ is a list $\bar{V}^{t}_{z_{i}}$, as mentioned in Table
III. The list $\bar{V}^{t}_{z_{i}}$ contains all the time points of the brain
slice $z_{i}$. Every 2D input image of the list $\tilde{I}_{z_{i}}^{t_{j}}$ at
time point $t_{j}$ is fed to a 2D encoder $E^{t_{j}}_{\text{2D-TCN}}$ to
extract features in the latent space. Each $E^{t_{j}}_{\text{2D-TCN}}$ encoder
returns a ($4\times 4\times\text{Ch}$) feature vector, where Ch corresponds to
the number of channels. The architecture merges the low-level feature vectors
across the different $t_{j}$ time points to capture the spatio-temporal
information. The merged feature vector
$\text{ETOT}_{\text{2D-TCN}}=[E^{t_{j}}_{\text{2D-TCN}}|\forall t_{j}\in t]$
is used as input to the TCN, which yields a one-dimensional vector
$O_{\text{2D-TCN}}$ of 64 elements. Finally, a decoder takes the
$O_{\text{2D-TCN}}$ and generates a final 2D image $P_{z_{i}}(x,y)$. The Dice
Coefficient loss (DCL), the same as the original paper, was implemented as the
loss function as follows:

$\text{DCL}(x,y)=\sum_{c}^{\mathcal{C}}\left(1-\frac{2\sum_{i}^{M\times
N}x_{i,c}y_{i,c}}{\sum_{i}^{M\times N}x_{i,c}+\sum_{i}^{M\times
N}y_{i,c}}\right)$

For more details about the _2D-TCN_ approach, we refer the reader to [41].

####  IV-A4 Approach 4: 3D Temporal Convolutional Network Single Encoder

We implemented a similar method from [42]. We call this approach _3D-TCN-SE_
due to using a single encoder (SE). Fig. 4 shows a simplified version of the
proposed architecture, emphasizing the input difference between the _2D-TCN_
and this model.

The _3D-TCN-SE_ model receives the 4D CTP scans re-sampled to 1 second per
time point. The input is a list of $t$ 3D volumes
$\bar{V}^{t}_{z_{\mathcal{I}}}$ (Table III). Each 3D input volume in the list
$\widehat{V}^{t_{j}}_{z_{\mathcal{I}}}(x,y)$ corresponds to the concatenation
of the $i$th brain slice $z_{i}$ plus its neighbouring slices $z_{i-1}$ and
$z_{i+1}$ over a specific time point $t_{j}$. The _3D-TCN-SE_ approach uses a
single encoder $E_{\text{3D-TCN}}$ for all the elements in the input list. It
is worth mentioning that the _3D-TCN-SE_ model is trained with the entire
brain images for comparison reasons and not with just the ipsilateral
hemisphere, as in the original paper [42].

(a) _3D-TCN_ architecture (Sec. IV-B1)

(b) _3D+time mJ-Net_ architecture (Sec. IV-B2)

(c) _4D mJ-Net_ architecture (Sec. IV-B3)

Figure 5: Visual comparison between the proposed architectures: (a) the
_3D-TCN_ , (b) the _3D+time mJ-Net_ , and (c) _4D mJ-Net_.

Figure 6: Illustration of the _3D+time mJ-Net_ model. The list of 2D+time
input
$\bar{V}^{t}_{z_{\mathcal{I}}}(x,y)=[\widehat{V}_{z_{i-1}},\widehat{V}_{z_{i}},\widehat{V}_{z_{i+1}}]$
is trained in parallel, where $z_{\mathcal{I}}=\\{z_{i-1},z_{i},z_{i+1}\\}$.
The output is a 2D image $P_{z_{i}}(x,y)$. The first max-pooling layer of each
block in the convolution section has a pool size of (2,1,1) to reduce the
first dimension by a factor of 2. The second max-pooling layer uses a pool
size of (3,1,1), while the third has a pool size of (5,1,1). The selection of
these pool sizes is due to reducing the time dimension. The remaining max-
pooling layers have a pool size of (2,1,1). The Attention layers utilize a
kernel of dimension 3 and a Leaky ReLU activation function. The 2D Upsampling
layers have an upsampling factor of 2. The last convolution layer has a kernel
of 1 and a Softmax activation function to produce a probability score for
every class.

###  IV-B Proposed 4D methods

All the proposed methods222The code is publicly available at
https://github.com/Biomedical-Data-Analysis-Laboratory/4D-mJ-Net use the
entire 4D CTP scan as input; the main difference lies in how the 4D input is
processed. The _3D-TCN_ is based on a _2D-TCN_ [41], modified to receive a
list of 3D input volumes. The _3D+time mJ-Net_ inputs a list of 2D+time brain
volumes from a CTP dataset, while the _4D mJ-Net_ uses the entire 4D structure
of a CTP dataset as input. Fig. 5 compares these architectures with their
respective inputs.

####  IV-B1 Approach 5: 3D Temporal Convolutional Network

We extend the architecture proposed by [41] for our application to exploit
further the information in the depth dimension. In the remainder of the paper,
we call our architecture _3D-TCN_. The main differences between the proposed
_3D-TCN_ and the _3D-TCN-SE_ (Sec. IV-A4) rely on the usage of a 3D encoder
for each input element, instead of a 3D single encoder, plus the possibility
to segment both core and penumbra regions, in comparison with segmenting only
the core areas.

The 4D CTP scans for the _3D-TCN_ are all re-sampled to 1 second per time
point. As described in Sec. IV-A3, the _3D-TCN_ architecture feeds each
element of the input list $\widehat{V}^{t_{j}}_{z_{\mathcal{I}}}$ at time
point $t_{j}$ to a specific 3D encoder $E^{t_{j}}_{\text{3D-TCN}}$ to extract
low-level features. Each $E^{t_{j}}_{\text{3D-TCN}}$ encoder returns a
$(4\times 4\times C)$ feature vector, where $C$ corresponds to the number of
channels. Each feature vector is merged to create a single input
$\text{ETOT}_{\text{3D-TCN}}=[E^{t_{j}}_{\text{3D-TCN}}|\forall t_{j}\in t]$.
The $\text{ETOT}_{\text{3D-TCN}}$ is used in the TCN, which generates a one-
dimensional vector $O_{\text{3D-TCN}}$ of 64 elements. The TCN’s output
$O_{\text{3D-TCN}}$ is then given in input to the decoder to create the final
predicted 2D image $P_{z_{i}}(x,y)$ of a brain slice $z_{i}$ at index $i$.

Figure 7: Illustration of the _4D mJ-Net_ architecture. The 4D input
$\widehat{V}^{t}_{z_{\mathcal{I}}}=\varphi(\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t})$
is the concatenation of a 2D+time volume $\widetilde{V}^{t}_{z_{i}}$ of a
brain slice $z_{i}$ at index $i$ over all the time points $t$ plus its
neighbouring brain slice volumes
($\widetilde{V}^{t}_{z_{i-1}},\widetilde{V}^{t}_{z_{i+1}}$). Two MonteCarlo
dropout layers [61] are added at the end of the 4D and 2D Convolution blocks.
The rate was set to $50\%$. These layers were added to reduce uncertainties in
the final predictions. The last convolution layer has a kernel of ($1\times
1$) and a Softmax activation function to produce a probability score for every
class.

####  IV-B2 Approach 6: 3D+time mJ-Net

We propose a model called _3D+time mJ-Net_ , an extension of the work of [51].
The proposed model inputs a list of 2D+time matrices; thus, the dimension of
this input can be defined as 3D+time. The input and output are presented in
Table III, whereas a visual example of the input for the model is given in
Fig. 3. Each element of the input list coincides with a possible input for the
_mJ-Net_ (details in Sec. IV-A2). $\bar{V}_{z_{\mathcal{I}}}^{t}$ consists of
a list of 2D+time volumes, where $z_{\mathcal{I}}=\\{z_{i-1},z_{i},z_{i+1}\\}$
is a set of brain slices containing the $i$th slice $z_{i}$ analyzed and its
neighboring slices $z_{i-1}$ and $z_{i+1}$. In case the index $i$ corresponds
to the first (or last) brain slice, $\widehat{V}^{t}_{z_{i-1}}$ (and
equivalently $\widehat{V}^{t}_{z_{i+1}}$) is set equal to
$\widehat{V}^{t}_{z_{i}}$. Every 2D+time volume from the input list
$\widehat{V}_{z_{i}}^{t}(x,y)$ is trained separately in the model through a
series of encoders ($E_{z_{i-1}},E_{z_{i}},E_{z_{i+1}}$) composed of 3D-Conv
and 2D-Conv layers. Each section is independent of the other: the convolution
layers have no shared weights.

Fig. 6 illustrates the model’s architecture. Attention layers [62] and 2D
Upsampling layers were implemented in the decoder section. Attention layers
benefit the architecture by focusing on target structures and help increase
the segmentation performances. With the sole exception of the last convolution
layer, each convolution layer uses a kernel of dimension 3 and a Leaky ReLU
activation function [63] with $\alpha=1/3$.

####  IV-B3 Approach 7: 4D mJ-Net

We propose another model called _4D mJ-Net_. We introduce this method to avoid
the three paths to process the 4D data presented in the previous architecture
(Sec. IV-B2). We still use a sliding window technique over the depth dimension
to simultaneously limit the amount of input data fed to the model, using three
consecutive brain slices at a time. Like the _3D+time mJ-Net_ model, also this
approach is an extension of the work of [51]. Information on this approach is
given in Table III. The 4D input tensor $\widehat{V}^{t}_{z_{\mathcal{I}}}$
contains both the time dimension and the neighboring slices of the $i$th brain
slice. The $\widehat{V}^{t}_{z_{\mathcal{I}}}$ is a concatenation of a 2D+time
volume $\widehat{V}^{t}_{z_{i}}$ of a brain slice $z_{i}$ at index $i$ over
all the time points $t$ together with its neighbouring 2D+time volumes
$\widehat{V}^{t}_{z_{i-1}},\widehat{V}^{t}_{z_{i+1}}$. This model can be
considered an early-fusion approach since the 4D input tensors
$\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t}$
are concatenated before being fed to the encoder’s model.

The proposed _4D mJ-Net_ model is a combination of both _3D+time mJ-Net_ (Sec.
IV-B2) and _mJ-Net_ (Sec. IV-A2). The proposed approach uses the same input
type that the _3D+time mJ-Net_ exploits. However, rather than a list of
2D+time volumes, the model concatenates the input into a single 4D tensor of
dimensions ($X\times Y\times Z\times T$).

Unlike 1D, 2D, and 3D Convolution layers, 4D Convolution layers are not
available in public DL frameworks (i.e., Keras333https://keras.io/ or
PyTorch444https://pytorch.org/). Thus, for this model, we implemented a novel
4D-Conv layer (details in Sec. III-C) which uses the convolutional layers
defined in the public DL frameworks to replicate a 4D convolution operation.

The architecture of the _4D mJ-Net_ is displayed in Fig. 7. No Attention
layers [62] were included in these models due to a considerable performance
decline. The output of the 4D-Conv layers is a tensor where the temporal
dimension has been squeezed and reduced; information are extrapolated from the
temporal dimension. Thus the output resulting from the 4D-Conv layers contains
only three dimensions ($X\times Y\times Z$) plus the channel dimension.
3D-Conv layers are implemented to reduce the depth dimension $Z$ and produce a
2D vector ($X\times Y$) plus the channel dimension.

A weighted categorical cross-entropy (WCC) loss [64] was the loss function
implemented for this method. The loss can be written as:

$\displaystyle\text{WCC}(x,y)=\sum_{c}^{\mathcal{C}}\sum_{i}^{M\times
N}(y_{i,c}\log x_{i,c})\cdot(w_{i,c}y_{i,c})$

where $w_{i,c}$ corresponds to the weight of the $i$th pixel for a class
$c\in\mathcal{C}$; $x_{i,c}$ is the $i$th predicted pixel, and $y_{i,c}$ is
the corresponding ground truth pixel.

###  IV-C Implementation details

TABLE III: Summary of the approaches.

Approach | Input | Output | Loss
---|---|---|---
Approach 1 [52]: _Multi-input PMs_ |  $\text{PMs}_{z_{i}}=[\text{CBF}_{z_{i}},\text{CBV}_{z_{i}}$, $\text{TMAX}_{z_{i}},\text{TTP}_{z_{i}},\text{MIP}_{z_{i}}]$ | $P_{z_{i}}(x,y)$ | FTL [58]
Approach 2 [51]: _mJ-Net_ |  $\widehat{V}_{z_{i}}^{t}(x,y)=\varphi(\tilde{I}_{z_{i}}^{t_{j}}(x,y)|\forall t_{j}\in t)$. | —"— | SDCL [59]
Approach 3 [41]: _2D-TCN_ |  $\bar{V}^{t}_{z_{i}}=[\tilde{I}_{z_{i}}^{t_{j}}|\forall t_{j}\in t]$ of 2D images $\tilde{I}_{z_{i}}^{t_{j}}$ | —"— | DCL
Approach 4 [42]: _3D-TCN-SE_ | $\bar{V}^{t}_{z_{\mathcal{I}}}=[\widehat{V}^{t_{j}}_{z_{\mathcal{I}}}|\forall t_{j}\in t]$ | —"— | SDCL [59]
Approach 5: _3D-TCN_ | —"— | —"— | —"—
Approach 6: _3D+time mJ-Net_ | $\bar{V}_{z_{\mathcal{I}}}^{t}(x,y)=[\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t}]$ | —"— | —"—
Approach 7: _4D mJ-Net_ | $\widehat{V}^{t}_{z_{\mathcal{I}}}=\varphi(\widehat{V}_{z_{i-1}}^{t},\widehat{V}_{z_{i}}^{t},\widehat{V}_{z_{i+1}}^{t})$ | —"— | WCC [64]

TABLE IV:  Experiment results for the validation set. Values in bold exhibit
the best results for each column and each class. Mean results plus standard
deviation for Dice Coefficient (DC), Hausdorff Distance (HD), and $\Delta V$
are presented. Results are for the penumbra and core areas divided by the
distinct patient groups (LVO, Non-LVO, WIS, and All). Note that for the DC,
higher values are better ($\Uparrow$), while for HD and $\Delta V$, lower
values are preferable ($\Downarrow$).

Method |  DC $\Uparrow$ |  HD (mm) $\Downarrow$ |  $\Delta V$ (ml) $\Downarrow$
---|---|---|---
LVO | Non-LVO | All | LVO | Non-LVO | All | LVO | Non-LVO | WIS | All
| Penumbra
_Multi-input PMs_ [52] | 0.70$\pm$0.1 | 0.27$\pm$0.3 | 0.47$\pm$0.3 | 2.9$\pm$0.4 | 1.4$\pm$0.7 | 2.0$\pm$0.8 | 27.0$\pm$28.6 | 10.0$\pm$15.5 | 9.8$\pm$8.1 | 19.0$\pm$24.2
_mJ-Net_ [51] | 0.66$\pm$0.2 | 0.39$\pm$0.3 | 0.50$\pm$0.3 | 2.9$\pm$0.5 | 2.6$\pm$0.6 | 2.7$\pm$0.7 | 25.5$\pm$20.0 | 24.7$\pm$29.2 | 45.5$\pm$39.1 | 27.2$\pm$26.0
_2D-TCN_ [41] | 0.12$\pm$0.1 | 0.02$\pm$0.0 | 0.07$\pm$0.1 | 4.1$\pm$0.5 | 3.8$\pm$0.6 | 4.0$\pm$0.6 | 81.3$\pm$65.6 | 80.6$\pm$57.8 | 131.6$\pm$93.1 | 86.0$\pm$66.5
_3D-TCN-SE_ [42] | 0.25$\pm$0.1 | 0.05$\pm$0.1 | 0.15$\pm$0.1 | 6.2$\pm$0.5 | 6.7$\pm$0.4 | 6.4$\pm$0.5 | 497.9$\pm$157.1 | 559.3$\pm$90.4 | 624.6$\pm$118.7 | 533.1$\pm$137.3
_3D-TCN_ | 0.23$\pm$0.1 | 0.04$\pm$0.1 | 0.14$\pm$0.1 | 4.3$\pm$0.4 | 4.4$\pm$0.5 | 4.3$\pm$0.5 | 85.3$\pm$64.0 | 142.7$\pm$51.4 | 164.2$\pm$43.8 | 114.2$\pm$65.3
_3D+time mJ-Net_ | 0.70$\pm$0.1 | 0.42$\pm$0.3 | 0.53$\pm$0.3 | 2.6$\pm$0.6 | 1.9$\pm$0.8 | 2.2$\pm$0.9 | 35.1$\pm$36.1 | 18.3$\pm$26.3 | 2.7$\pm$2.6 | 25.7$\pm$32.4
_4D mJ-Net_ | 0.66$\pm$0.1 | 0.44$\pm$0.3 | 0.51$\pm$0.3 | 2.3$\pm$0.6 | 1.3$\pm$0.7 | 1.7$\pm$1.0 | 41.4$\pm$37.2 | 6.1$\pm$6.3 | 0.0$\pm$0.0 | 24.3$\pm$32.9
| Core
_Multi-input PMs_ [52] | 0.37$\pm$0.3 | 0.21$\pm$0.3 | 0.28$\pm$0.3 | 1.2$\pm$0.8 | 0.4$\pm$0.4 | 0.8$\pm$0.8 | 9.4$\pm$20.3 | 0.8$\pm$1.3 | 0.5$\pm$0.5 | 5.3$\pm$15.3
_mJ-Net_ [51] | 0.27$\pm$0.2 | 0.21$\pm$0.2 | 0.22$\pm$0.2 | 1.5$\pm$0.7 | 0.8$\pm$0.6 | 1.2$\pm$0.6 | 5.5$\pm$4.9 | 1.0$\pm$1.2 | 1.0$\pm$1.1 | 3.4$\pm$4.3
_2D-TCN_ [41] | 0.02$\pm$0.0 | 0.01$\pm$0.0 | 0.01$\pm$0.0 | 1.9$\pm$0.7 | 1.5$\pm$0.6 | 1.7$\pm$0.7 | 11.8$\pm$13.3 | 8.1$\pm$8.2 | 11.0$\pm$11.2 | 10.3$\pm$11.4
_3D-TCN-SE_ [42] | 0.00$\pm$0.0 | 0.00$\pm$0.0 | 0.00$\pm$0.0 | 1.2$\pm$0.9 | 0.4$\pm$0.4 | 0.8$\pm$0.8 | 12.7$\pm$15.6 | 1.9$\pm$2.8 | 0.0$\pm$0.0 | 7.5$\pm$12.8
_3D-TCN_ | 0.02$\pm$0.0 | 0.01$\pm$0.0 | 0.01$\pm$0.0 | 1.4$\pm$0.8 | 0.8$\pm$0.4 | 1.1$\pm$0.7 | 12.0$\pm$14.3 | 1.9$\pm$2.1 | 2.4$\pm$1.9 | 7.3$\pm$11.6
_3D+time mJ-Net_ | 0.21$\pm$0.2 | 0.12$\pm$0.2 | 0.16$\pm$0.4 | 1.1$\pm$0.7 | 0.4$\pm$0.4 | 0.7$\pm$0.7 | 8.1$\pm$10.6 | 1.3$\pm$1.6 | 0.0$\pm$0.0 | 4.8$\pm$8.5
_4D mJ-Net_ | 0.29$\pm$0.2 | 0.21$\pm$0.2 | 0.23$\pm$0.2 | 1.6$\pm$0.9 | 0.5$\pm$0.4 | 1.0$\pm$0.9 | 25.9$\pm$37.0 | 1.4$\pm$2.2 | 0.0$\pm$0.0 | 14.3$\pm$29.6

Table III provides information about all the methods. All the methods
mentioned in Sec. IV-A and Sec. IV-B utilize Adam as the optimizer [65] with a
learning rate of 0.0003 and a step-based decay rate of 0.95 every ten epochs.
The batch size is set to 2. An early stopping function is called if there is
no decrement in the validation loss after 25 epochs. During training, L1 and
L2 regularizations are applied in the kernels, plus a max norm constraint is
also used in the kernel and bias weights. All experiments were implemented in
Python using Keras (2.3.1) with Tensorflow as the backend and trained using an
NVIDIA Tesla V100 GPU (32 GB memory).

##  V Experiments & Results

We assess the proposed methods on a local dataset of CTP scans from 152
patients (Sec. II). All experiments are performed with the same training set
and evaluated over the validation set (details in Table I). The test set is
used only to make predictions with the best models with CTP scans that the
methods have not seen before. Since the Non-LVO group has smaller ischemic
areas than the LVO patients, we set a higher penalty for every
misclassification of penumbra and core classes for this sub-group during
training.

###  V-A Evaluation metrics

Three evaluation metrics are used to assess the various experiments’ models.
The Dice Coefficient (DC), the Hausdorff Distance (HD) [66], and the absolute
difference in the volumes ($\Delta V$). We employ the DC to compare the model
predictions with the ground truth segmentations. The DC between two
segmentations $x$ and $y$ is given by the following equation:

$\displaystyle\text{DC}(x,y)=2\frac{|x\cap y|}{|x|+|y|}$

where the range for the DC is $[0,1]$; thus a $\text{DC}(x,y)=1$ corresponds
to a perfect match between the prediction $x$ and ground truth $y$
segmentations.

The HD measures how two subsets $(\mathcal{A},\mathcal{B})$ are distant from
each other, and it is formulated as follows:

$\displaystyle\text{HD}(\mathcal{A},\mathcal{B})=\max\left\\{\,h(\mathcal{A},\mathcal{B}),\,h(\mathcal{B},\mathcal{A})\,\right\\}$

where
$h(\mathcal{A},\mathcal{B})=\max_{a\in\mathcal{A}}\min_{b\in\mathcal{B}}||a-b||$.
The range value for the HD is $[0,\infty]$.

The absolute difference in the volumes $\Delta V$ between the prediction
volume $V_{x}$ and the ground truth volume $V_{y}$ can be expressed as:

$\displaystyle\Delta V(V_{y},V_{x})=|V_{y}-V_{x}|$

The range for $\Delta V$ is $[0,\infty]$, and $\Delta V(V_{y},V_{x})=0$
represents a perfect match between the two volumes. The $\Delta
V(V_{y},V_{x})$ is an essential evaluation metric for the WIS group due to the
lack of ground truth segmentations in this group. The other metrics are not
suitable for understanding how the predictions will be since the ground truth
will always be empty.

The best scenario for a model is to produce high DC with low HD and $\Delta
V$: this implies a strong correlation between the predicted areas and the
ground truth regions. If the results show high $\Delta V$ (or HD) with low DC,
an over-segmentation of the ischemic areas is perceived. On the other hand,
promising outcomes of $\Delta V$ (or HD) with mediocre DC results imply an
under-segmentation of the predicted regions.

TABLE V: Ablation study for the _4D mJ-Net_ model showing how various pre-
processing steps (HE, $\gamma$, $z$) and re-sampling ($\uplus$) affect the
Dice Coefficient (DC) for the validation results. Penumbra and core DC scores
are shown for all the classes together. Note that for the DC, higher values
are better ($\Uparrow$).

Ablation Setting |  DC $\Uparrow$
---|---
|  |  | LVO | Non-LVO
HE | $\gamma$ | $z$ | $\uplus$ | Penumbra | Core | Penumbra | Core
- | - | - | - |  0.42$\pm$0.2 |  0.25$\pm$0.2 |  0.20$\pm$0.2 |  0.16$\pm$0.2
✓ | - | - | - |  0.32$\pm$0.3 |  0.24$\pm$0.2 |  0.13$\pm$0.2 |  0.16$\pm$0.3
- | ✓ | - | - |  0.00$\pm$0.0 |  0.07$\pm$0.1 |  0.00$\pm$0.0 |  0.06$\pm$0.1
✓ | ✓ | - | - |  0.48$\pm$0.2 |  0.28$\pm$0.2 |  0.24$\pm$0.2 |  0.20$\pm$0.3
- | - | ✓ | - |  0.01$\pm$0.0 |  0.14$\pm$0.2 |  0.01$\pm$0.0 |  0.08$\pm$0.1
✓ | - | ✓ | - |  0.53$\pm$0.2 |  0.22$\pm$0.2 |  0.35$\pm$0.3 |  0.12$\pm$0.2
- | ✓ | ✓ | - |  0.28$\pm$0.2 |  0.17$\pm$0.2 |  0.08$\pm$0.1 |  0.05$\pm$0.1
✓ | ✓ | ✓ | - | 0.66$\pm$0.1 | 0.29$\pm$0.2 | 0.44$\pm$0.3 | 0.21$\pm$0.2
- | - | - | ✓ |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0
✓ | - | - | ✓ |  0.07$\pm$0.1 |  0.26$\pm$0.2 |  0.01$\pm$0.0 |  0.05$\pm$0.1
- | ✓ | - | ✓ |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0
✓ | ✓ | - | ✓ |  0.41$\pm$0.3 |  0.29$\pm$0.2 |  0.11$\pm$0.2 |  0.12$\pm$0.2
- | - | ✓ | ✓ |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0
✓ | - | ✓ | ✓ |  0.56$\pm$0.2 |  0.24$\pm$0.2 |  0.37$\pm$0.3 |  0.18$\pm$0.2
- | ✓ | ✓ | ✓ |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0 |  0.00$\pm$0.0
✓ | ✓ | ✓ | ✓ |  0.59$\pm$0.2 | 0.29$\pm$0.2 |  0.40$\pm$0.3 |  0.20$\pm$0.2

TABLE VI: Inter-observer variability results for test set. Values are
presented for the two best-proposed architectures (_3D+time mJ-Net_ , _4D mJ-
Net_) in relation to manual annotations generated separately by two expert
neuroradiologists ($\text{NR}_{1}$, $\text{NR}_{2}$) over the test set. An
investigation of the inter-observer variability between $\text{NR}_{1}$ and
$\text{NR}_{2}$ is performed (last row or each class). Note that for the DC,
higher values are better ($\Uparrow$), while for HD and $\Delta V$, lower
values are preferable ($\Downarrow$).

Method |  DC $\Uparrow$ |  HD (mm) $\Downarrow$ |  $\Delta V$ (ml) $\Downarrow$
---|---|---|---
LVO | Non-LVO | All | LVO | Non-LVO | All | LVO | Non-LVO | WIS | All
| Penumbra
_3D+time mJ-Net_ vs ($\text{NR}_{1}$ vs $\text{NR}_{2}$) | 0.70$\pm$0.1 | 0.32$\pm$0.3 | 0.51$\pm$0.3 | 2.6$\pm$0.5 | 1.5$\pm$0.6 | 2.1$\pm$0.9 | 36.7$\pm$36.4 | 6.5$\pm$4.9 | 10.1$\pm$3.1 | 24.2$\pm$31.1
_4D mJ-Net_ vs ($\text{NR}_{1}$ vs $\text{NR}_{2}$) | 0.67$\pm$0.1 | 0.25$\pm$0.3 | 0.47$\pm$0.3 | 2.4$\pm$0.5 | 0.9$\pm$0.5 | 1.7$\pm$1.0 | 34.1$\pm$30.6 | 5.3$\pm$6.6 | 0.0$\pm$0.0 | 21.4$\pm$27.7
$\text{NR}_{1}$ vs $\text{NR}_{2}$ | 0.78$\pm$0.1 | 0.65$\pm$0.2 | 0.67$\pm$0.2 | 2.2$\pm$0.4 | 0.8$\pm$0.6 | 1.6$\pm$1.0 | 33.3$\pm$27.7 | 5.5$\pm$9.2 | 0.0$\pm$0.0 | 21.0$\pm$25.9
| Core
_3D+time mJ-Net_ vs ($\text{NR}_{1}$ vs $\text{NR}_{2}$) | 0.19$\pm$0.2 | 0.01$\pm$0.0 | 0.12$\pm$0.2 | 1.6$\pm$0.8 | 0.3$\pm$0.4 | 1.0$\pm$1.0 | 14.6$\pm$18.2 | 0.9$\pm$2.3 | 0.0$\pm$0.0 | 8.7$\pm$15.4
_4D mJ-Net_ vs ($\text{NR}_{1}$ vs $\text{NR}_{2}$) | 0.28$\pm$0.2 | 0.03$\pm$0.1 | 0.18$\pm$0.2 | 1.8$\pm$0.8 | 0.3$\pm$0.5 | 1.1$\pm$1.0 | 21.2$\pm$31.7 | 1.8$\pm$4.5 | 0.0$\pm$0.0 | 12.8$\pm$25.9
$\text{NR}_{1}$ vs $\text{NR}_{2}$ | 0.44$\pm$0.2 | 0.15$\pm$0.2 | 0.30$\pm$0.3 | 1.4$\pm$0.6 | 0.2$\pm$0.4 | 0.9$\pm$0.8 | 5.6$\pm$4.3 | 0.7$\pm$1.9 | 0.0$\pm$0.0 | 3.5$\pm$4.2

###  V-B Comparison with other methods

The proposed _3D+time mJ-Net_ , _4D mJ-Net_ , and _3D-TCN_ methods are
compared with alternative models: the _2D-TCN_ [41], the _mJ-Net_ [51], the
_3D-TCN-SE_ [42], and the _Multi-input PMs_ [52].

Table IV presents the evaluation metrics’ results over the validation set.
Results are presented for each group distinctly (LVO, Non-LVO, WIS, and all)
to highlight the strengths and weaknesses of each model over the various
groups composing the dataset. An extensive number of experiments are performed
for all the analyzed models. However, to present a fair comparison among the
various models, we only introduce the methods with a combination of parameters
that yield the best results omitting the other combinations tested during
experiments. Qualitative comparison results of random brain slices extracted
from the validation set are provided in Fig. 8.

###  V-C Ablation Study

To demonstrate the effects of the pre-processing steps (Sec. III-B), we
conduct an ablation study on the _4D mJ-Net_ architecture. Moreover, we re-
sampled the CTP scans to handle the irregular temporal dimension and studied
the effect of using re-sampled scans during the model’s training. Different CT
scan vendors have different imaging acquisition protocols, thus re-sampling
the scans to a fixed time-sampling-rate is an reasonable step to increase the
versatility and usability across hospitals. DC are illustrated in Table V
showing performances of the network for all the groups trained with the
datasets using different types of pre-processing steps and re-sampled scans.
The study aims to systematically analyze the contribution of each pre-
processing step toward improving the overall results. We begin by defining a
baseline configuration consisting of the raw input images without pre-
processing (first row in Table V). Subsequently, we incrementally introduce
and evaluate individual pre-processing steps, such as histogram equalization
(HE), gamma correction $(\gamma)$, and z-score ($z$).

###  V-D Inter-observer variability

Two expert neuroradiologists ($\text{NR}_{1}$, $\text{NR}_{2}$) manually
annotated the scans of 33 randomly selected patients: 19 from the LVO group,
11 from the Non-LVO, and 3 from the WIS subset. The manual annotation images
were generated using the same criteria endorsed for creating the ground truth
images, as explained in Sec. II-A. An investigation of the inter-observer
variability between $\text{NR}_{1}$, $\text{NR}_{2}$, and the two best-
proposed models is presented in Table VI.

Figure 8: Qualitative comparisons for the tested models. The brain slices are
taken randomly from distinct patients from the validation set, divided by
group (left to right). Each row represents the results for each model involved
in the study.

##  VI Discussion

Early detection and intervention in AIS patients are of vital importance [67,
68, 69]. In this study, we have proposed different architectures to utilize
the 4D CTP input to use the spatio-temporal information better than in
existing approaches. We suggest expanding the _mJ-Net_ and showing two ways of
segmenting ischemic areas in patients suspected of AIS. In addition, we expand
another method (_3D-TCN_) for comparison reasons. We use the entire raw 4D CTP
data and feed different combinations as input to our proposed approaches to
prevent possible loss of spatio-temporal information. Studying the data as an
independent volume and neglecting its spatio-temporal nature can lead to the
loss of relevant information. All proposed approaches return a series of 2D
segmented images as output, later stacked together to produce a 3D volume.
Returning a list of 2D images as output is less computationally expensive and
less memory intensive than directly returning 3D volumetric data as output
[70].

Few studies have adopted 4D datasets in DNN models to detect ischemic lesions
in patients affected by a stroke [7, 41, 43, 42]. This is rooted in the high
computational complexity of 4D data and the lack of ground truth for the whole
set. The limitations that these approaches encounter are as follows: 1)
datasets used for the training and evaluation take into account only a subset
of the entire population; 2) segmentations are only performed on the core
areas, excluding penumbra regions; 3) ground truth images derived from follow-
up DWI or NCCT present some limitations [45, 46].

To our knowledge, this is the first study using 4D CTP data to segment both
the ischemic regions, penumbra and core. Additionally, we include data from
all patients, regardless of stroke severity, to train our models. Rather than
entrusting ground truth images from follow-up DWI or NCCT studies that are
usually taken 24 hours or several days after the onset of stroke, our proposed
methods were trained with ground truth images obtained from the CTP captured
at admission, PMs, and follow-up scans (Sec. II-A).

We use three evaluation metrics to assess the models’ performances: DC, HD,
and $\Delta V$ compared with our previously developed algorithms and other
state-of-the-art algorithms. Results in Table IV demonstrate that increasing
the input dimension benefits achieving more precise segmentation, especially
for the Non-LVO and WIS groups, regardless of the class. Thus, when a smaller
portion of the brain is affected, the whole dataset’s usage helps achieve
better segmentation results. The ablation study (Table V) shows how including
the pre-processing steps and not re-sampling the CTP scans helped improve the
overall segmentation performances. It is worth mentioning that using the pre-
processing steps and re-sampled CTP scans yields the second best overall
results ( last row in Table V), establishing the validity of the pre-
processing sequence.

Visual results of random validation brain slices are shown in Fig. 8, where we
can see that our proposed approaches (_3D+time mJ-Net_ , _4D mJ-Net_) are less
prone to over-segment, especially in the Non-LVO and WIS groups. It is
reported that LVO cases are less common compared to Non-LVO. On average, LVOs
are estimated to represent around 30% of all AIS cases [71]. Thus, a neural
network that can accurately segment patients in the Non-LVO group can be
valuable in a real-life scenario. Nonetheless, patients with LVO represent a
clinically significant proportion of patients presenting with AIS, especially
considering the grim natural course of the disease.

The results presented in Table IV indicate that all _mJ-Net_ models have
improved where the input data dimension has increased, regardless of the
patients’ group. Fig. 8 shows that using 2D+time input for the _mJ-Net_ [51]
led to over-segmentation of penumbra class in separate brain tissue sections,
brain slice 4-6. The visual results for the Non-LVO and WIS groups highlight
the limitations of this model: the over-segmentation of the penumbra regions
might affect the usage in a real-life scenario, and an overestimation of the
penumbra area can generate uncertainties for treatment decisions.

Adding depth as an extra dimension to the input of models (_3D+time mJ-Net_
and _4D mJ-Net_) determines an increment in the performances for both classes
in the three patient groups. A significant increase is noticeable for the DC
metric in the Non-LVO group, regardless of the class. An essential difference
between these two architectures is how they exploit their structures’ input.
The _3D+time mJ-Net_ is considered a late-fusion approach as the data sources
are used independently and fused close to decision-making. Statistical results
presented for the _3D+time mJ-Net_ show promising general performances for the
LVO group. However, an underestimation of the core class, regardless of the
patient group, can be noticed from the visual results in Fig. 8 and the low HD
metric. Nevertheless, _3D+time mJ-Net_ achieved the best HD for the core class
in all the groups. The _3D+time mJ-Net_ can precisely segment ischemic regions
with large areas, as shown by the first three slices in Fig. 8. This can also
be manifested in the high DC score achieved for the LVO group for both classes
(Table IV).

The _4D mJ-Net_ network has learned to precisely segment the ischemic regions
even without re-sampling the CTP scans, as shown in Table V. The model fuses
the data before they are fed to the network. Visual results in Fig. 8 and
values in Table IV indicate that the _4D mJ-Net_ model segments the penumbra
class more precisely compared to the other approaches that use raw CTP as
their input. This promising performance follows in all patient groups. The _4D
mJ-Net_ achieved the highest DC metric for core and penumbra regions in
patients with Non-LVO. This approach gives the best HD for penumbra in all the
groups. The _4D mJ-Net_ showed high precision in detecting small ischemic
areas, as shown in sample brain slices 3 to 5 in Fig. 8. Furthermore, the _4D
mJ-Net_ model can correctly predict no ischemic regions in WIS patients, as
demonstrated by the results for the $\Delta V$ in the WIS group. However, it
over-segments the core class in patients with LVO. This means that including
the complete spatio-temporal information of the data and following an early
fusion approach leads to better prediction in Non-LVO and WIS groups, where
small areas are of interest.

Models based on TCN, in general, showed poor results statistically in Table IV
and visually in Fig. 8. They extremely over-segment the penumbra class and
poorly segment the core class. The original _2D-TCN_ and _3D-TCN-SE_ were
designed to segment only one class, the ischemic core. This can explain the
poor performance of segmenting the two classes. Besides, in [42] (_3D-TCN-
SE_), the model was trained to use only the ipsilateral hemisphere. For a fair
comparison, the model’s training was done over both hemispheres, which can
cause over-segmentation in penumbra regions.

As the name indicates, the _Multi-input PMs_ model [52] takes parametric maps
and pre-processed data obtained from CTP scans. The experiment results of this
model show a high DC value for the penumbra class in the LVO group, as also
seen in the first three brain slices of Fig. 8. This highlights that this
method presents satisfactory results for large ischemic areas. However, when
the region’s volume is small or vacant, the predictions are not optimal: see
brain slices 4 and 6 in Fig. 8. Although HD and $\Delta V$ are encouraging, DC
values show under-segmentation in the core and penumbra classes for the Non-
LVO set. Using PMs derived from CTP scans limits the machine to only learn
from specific pre-processed information.

The inter-observer variability results, highlighted in Table VI, show
promising outcomes for the proposed methods with the results achieved by the
two expert neuroradiologists ($\text{NR}_{1}$, $\text{NR}_{2}$). Similar
statistic values can be observed between $\text{NR}_{1}$ vs. $\text{NR}_{2}$
and the _4D mJ-Net_ for the penumbra class in connection with the LVO group.
The same results as the neuroradiologists were achieved by the _4D mJ-Net_ for
the $\Delta V$ in the WIS group. The proposed _3D+time mJ-Net_ model produces
higher results for the DC compared to the _4D mJ-Net_ in association with the
penumbra class. However, the results for the core regions could be more
satisfactory. The inter-observer variability outcomes for the HD and $\Delta
V$ highlight substantial similarity among the proposed approaches and the
neuroradiologists, except for the core class connected with the LVO group. The
difference can be due to an over-segmentation of this particular class, which
can be highly complex to detect for the models considering its small size.

###  VI-A Common limitations

All the assessed approaches have faced general limitations. The images used
during the training of each model are from CT scanners of the same vendor.
This causes a lack of diversity in the data. The annotations used as ground
truth surround the essential ischemic regions (penumbra and core) but do not
represent the areas perfectly. They might leave out small parts of the core
spread into the penumbra tissue and details of the penumbra misclassified as
healthy brain tissue [51].

##  VII Conclusions

Fast and precise diagnosis and treatment are of vital importance in AIS
patients. In this paper, we proposed to use 4D CTP as input to extract spatio-
temporal information for segmenting core and penumbra areas in patients with
AIS. This is presented primarily by expanding the _mJ-Net_ in two ways.
Furthermore, we introduced a novel 4D-Conv layer to exploit spatio-temporal
information. Two of our approaches (_3D+time mJ-Net_ and _4D mJ-Net_) achieved
promising results for all the classes involved. _3D+time mJ-Net_ can precisely
delineates large ischemic areas, while underestimating the core class. Our
best network (_4D mJ-Net_) can correctly segments penumbra regions, regardless
of patient groups with a 0.53 DC score on average. However, with an average of
0.23 DC score, it overestimates the core class for the LVO group.

We used the entire 4D CTP dataset of all patients and compared models using
different input types. We demonstrated that relying only on images derived
from the CTP scans (i.e., PMs) or on a restricted number of dimensions (i.e.,
2D, 2D+time, 3D) limits the prediction accuracy in DNN-based approaches.
Moreover, we segmented both penumbra and core regions in ischemic brain tissue
since an accurate and fast understanding of both is essential for fast
treatment decisions in AIS.

Further studies with larger datasets, including images from different vendors
and various acquisition parameters, are still needed to validate our methods.
The ISLES18 dataset [72] can be used in future work; the dataset uses FIAs as
ground truth labels, which is not in the scope of the aforementioned
architectures, thus some changes must be implemented for validating the
methods. Due to complex and time-consuming work for manual annotations,
further work on optimizing the segmentation using unsupervised neural networks
is encouraged.

## Acknowledgment

We declare that we do not have any commercial or associative interest that
represents a conflict of interest in connection with the work submitted.

## References

  * [1] Jeans Astrup, Bo K Siesjö and Lindsay Symon  “Thresholds in cerebral ischemia-the ischemic penumbra.”  In _Stroke_ 12.6 Am Heart Assoc, 1981, pp. 723–725
  * [2] European Stroke Organisation (ESO) Executive Committee and ESO Writing Committee  “Guidelines for management of ischaemic stroke and transient ischaemic attack 2008”  In _Cerebrovascular diseases_ 25.5 Karger Publishers, 2008, pp. 457–507
  * [3] Bruce CV Campbell et al.  “CT perfusion improves diagnostic accuracy and confidence in acute ischaemic stroke”  In _Journal of Neurology, Neurosurgery & Psychiatry_ 84.6 BMJ Publishing Group Ltd, 2013, pp. 613–618
  * [4] A Bivard et al.  “Defining acute ischemic stroke tissue pathophysiology with whole brain CT perfusion”  In _Journal of Neuroradiology_ 41.5 Elsevier, 2014, pp. 307–315
  * [5] KD Kurz et al.  “Radiological imaging in acute ischaemic stroke”  In _European journal of neurology_ 23 Wiley Online Library, 2016, pp. 8–17
  * [6] Kohsuke Kudo et al.  “Differences in CT perfusion maps generated by different commercial software: quantitative analysis by using identical source data of acute stroke patients”  In _Radiology_ 254.1 Radiological Society of North America, Inc., 2010, pp. 200–209
  * [7] Mohsen Soltanpour et al.  “Using End-to-End imbalanced Temporal Deep Gaussian Process (iTDGP) to Predict Acute Ischemic Stroke Lesions”, 2022
  * [8] Christopher D d’Esterre et al.  “Time-dependent computed tomographic perfusion thresholds for patients with acute ischemic stroke”  In _Stroke_ 46.12 Am Heart Assoc, 2015, pp. 3390–3397
  * [9] Longting Lin et al.  “Whole-brain CT perfusion to quantify acute ischemic penumbra and core”  In _Radiology_ 279.3 Radiological Society of North America, 2016, pp. 876–887
  * [10] Ferghal McVerry et al.  “Derivation and evaluation of thresholds for core and tissue at risk of infarction using CT perfusion”  In _Journal of neuroimaging_ 24.6 Wiley Online Library, 2014, pp. 562–568
  * [11] Carlo W Cereda et al.  “A benchmarking tool to evaluate computer tomography perfusion infarct core predictions against a DWI standard”  In _Journal of Cerebral Blood Flow & Metabolism_ 36.10 SAGE Publications Sage UK: London, England, 2016, pp. 1780–1789
  * [12] Anne Nielsen, Mikkel Bo Hansen, Anna Tietze and Kim Mouridsen  “Prediction of tissue outcome and assessment of treatment effect in acute ischemic stroke using deep learning”  In _Stroke_ 49.6 Am Heart Assoc, 2018, pp. 1394–1401
  * [13] Andrew Bivard et al.  “Does variability in automated perfusion software outputs for acute ischemic stroke matter? Reanalysis of EXTEND perfusion imaging”  In _CNS neuroscience & therapeutics_ 28.1 Wiley Online Library, 2022, pp. 139–144
  * [14] Chi-Long Chen et al.  “An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning”  In _Nature communications_ 12.1 Nature Publishing Group, 2021, pp. 1–13
  * [15] Neel Kanwal et al.  “Quantifying the effect of color processing on blood and damaged tissue detection in Whole Slide Images”  In _IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP 2022)_ , 2022, pp. 1–5  DOI: 10.1109/IVMSP54334.2022.9816283
  * [16] Saul Fuster et al.  “Invasive Cancerous Area Detection in Non-Muscle Invasive Bladder Cancer Whole Slide Images”  In _2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP)_ , 2022, pp. 1–5  DOI: 10.1109/IVMSP54334.2022.9816352
  * [17] Neel Kanwal et al.  “The Devil is in the Details: Whole Slide Image Acquisition and Processing for Artifacts Detection, Color Variation, and Data Augmentation: A Review”  In _IEEE Access_ 10 IEEE, 2022, pp. 58821–58844
  * [18] Neel Kanwal and al.  “Vision Transformers for Small Histological Datasets learned through Knowledge Distillation”  In _Advances in Knowledge Discovery and Data Mining: 27th Pacific-Asia Conference, Osaka, Japan, May 25–28, 2023_ , 2023, pp. in press  Springer Nature
  * [19] Ishani Chakraborty, Ahmed Elgammal and Randall S Burd  “Video based activity recognition in trauma resuscitation”  In _2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG)_ , 2013, pp. 1–8  IEEE
  * [20] Øyvind Meinich-Bache et al.  “Activity recognition from newborn resuscitation videos”  In _IEEE journal of biomedical and health informatics_ 24.11 IEEE, 2020, pp. 3258–3267
  * [21] Fatma Patlar Akbulut, Baris Ikitimur and Aydin Akan  “Wearable sensor-based evaluation of psychosocial stress in patients with metabolic syndrome”  In _Artificial Intelligence in Medicine_ 104 Elsevier, 2020, pp. 101824
  * [22] Tasweer Ahmed, Syed Tahir Hussain Rizvi and Neel Kanwal  “Transforming Spatio-Temporal Self-Attention Using Action Embedding for Skeleton-Based Action Recognition”  In _Available at SSRN 4379239_ , 2023
  * [23] Oliver Amft and Gerhard Tröster  “Recognition of dietary activity events using on-body sensors”  In _Artificial Intelligence in Medicine_ 42.2 Elsevier, 2008, pp. 121–136
  * [24] Olaf Ronneberger, Philipp Fischer and Thomas Brox  “U-net: Convolutional networks for biomedical image segmentation”  In _International Conference on Medical image computing and computer-assisted intervention_ , 2015, pp. 234–241  Springer
  * [25] Fabian Isensee et al.  “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation”  In _Nature methods_ 18.2 Nature Publishing Group, 2021, pp. 203–211
  * [26] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh and Jianming Liang  “UNet++: A nested u-net architecture for medical image segmentation”  In _Deep learning in medical image analysis and multimodal learning for clinical decision support_ Springer, 2018, pp. 3–11
  * [27] Rune Wetteland et al.  “A multiscale approach for whole-slide image segmentation of five tissue classes in urothelial carcinoma slides”  In _Technology in Cancer Research & Treatment_ 19 SAGE Publications Sage CA: Los Angeles, CA, 2020, pp. 1533033820946787
  * [28] Rayyan Azam Khan, Yigang Luo and Fang-Xiang Wu  “RMS-UNet: Residual multi-scale UNet for liver and lesion segmentation”  In _Artificial Intelligence in Medicine_ 124 Elsevier, 2022, pp. 102231
  * [29] Konstantinos Kamnitsas et al.  “Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation”  In _Medical image analysis_ 36 Elsevier, 2017, pp. 61–78
  * [30] Albert Clèrigues et al.  “Acute ischemic stroke lesion core segmentation in CT perfusion images using fully convolutional neural networks”  In _Computers in Biology and Medicine_ 115 Elsevier, 2019, pp. 103487  DOI: 10.1016/j.compbiomed.2019.103487
  * [31] S Mazdak Abulnaga and Jonathan Rubin  “Ischemic stroke lesion segmentation in CT perfusion scans using pyramid pooling and focal loss”  In _International MICCAI Brainlesion Workshop_ , 2018, pp. 352–363  Springer  DOI: 10.1007/978-3-030-11723-8“˙36
  * [32] Ryan A Rava et al.  “Investigation of convolutional neural networks using multiple computed tomography perfusion maps to identify infarct core in acute ischemic stroke patients”  In _Journal of Medical Imaging_ 8.1 International Society for OpticsPhotonics, 2021, pp. 014505
  * [33] Mohsen Soltanpour, Russell Greiner, Pierre Boulanger and Brian Buck  “Ischemic stroke lesion prediction in CT perfusion scans using multiple parallel U-nets following by a pixel-level classifier”  In _2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE)_ , 2019, pp. 957–963  IEEE
  * [34] Mohsen Soltanpour, Russ Greiner, Pierre Boulanger and Brian Buck  “Improvement of automatic ischemic stroke lesion segmentation in CT perfusion maps using a learned deep neural network”  In _Computers in Biology and Medicine_ 137 Elsevier, 2021, pp. 104849
  * [35] Aimen S Kasasbeh et al.  “Artificial Neural Network Computer Tomography Perfusion Prediction of Ischemic Core”  In _Stroke_ 50.6 Am Heart Assoc, 2019, pp. 1578–1581  DOI: 10.1161/STROKEAHA.118.022649
  * [36] Luca Tomasetti et al.  “Self-Supervised Few-Shot Learning for Ischemic Stroke Lesion Segmentation”  In _arXiv preprint arXiv:2303.01332_ , 2023
  * [37] Freda Werdiger et al.  “Machine learning segmentation of core and penumbra from acute stroke CT perfusion data”  In _Frontiers in neurology_ 14 Frontiers Media SA, 2023
  * [38] Lucas Vries et al.  “PerfU-Net: Baseline infarct estimation from CT perfusion source data for acute ischemic stroke”  In _Medical Image Analysis_ Elsevier, 2023, pp. 102749
  * [39] Jeroen Bertels, David Robben, Dirk Vandermeulen and Paul Suetens  “Contra-lateral information CNN for core lesion segmentation based on native CTP in acute stroke”  In _Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I 4_ , 2019, pp. 263–270  Springer
  * [40] Ezequiel Rosa et al.  “Differentiable deconvolution for improved stroke perfusion analysis”  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VII 23_ , 2020, pp. 593–602  Springer
  * [41] Kimberly Amador et al.  “Stroke Lesion Outcome Prediction Based on 4D CT Perfusion Data Using Temporal Convolutional Networks”  In _Medical Imaging with Deep Learning_ , 2021, pp. 22–33  PMLR
  * [42] Kimberly Amador et al.  “Predicting treatment-specific lesion outcomes in acute ischemic stroke from 4D CT perfusion imaging using spatio-temporal convolutional neural networks”  In _Medical Image Analysis_ Elsevier, 2022, pp. 102610
  * [43] David Robben et al.  “Prediction of final infarct volume from native CT perfusion and treatment parameters using deep learning”  In _Medical image analysis_ 59 Elsevier, 2020, pp. 101589
  * [44] Amber Bucker et al.  “Associations of ischemic lesion volume with functional outcome in patients with acute ischemic stroke: 24-hour versus 1-week imaging”  In _Stroke_ 48.5 Am Heart Assoc, 2017, pp. 1233–1240
  * [45] PD Schellinger et al.  “Evidence-based guideline: the role of diffusion and perfusion MRI for the diagnosis of acute ischemic stroke: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology”  In _Neurology_ 75.2 American Academy of Neurology, 2010, pp. 177
  * [46] Mayank Goyal et al.  “Challenging the ischemic core concept in acute ischemic stroke imaging”  In _Stroke_ 51.10 Am Heart Assoc, 2020, pp. 3147–3155
  * [47] Chelsea S Kidwell et al.  “Thrombolytic reversal of acute human cerebral ischemic injury shown by diffusion/perfusion magnetic resonance imaging”  In _Annals of Neurology: Official Journal of the American Neurological Association and the Child Neurology Society_ 47.4 Wiley Online Library, 2000, pp. 462–469
  * [48] Marc-Antoine Labeyrie et al.  “Diffusion lesion reversal after thrombolysis: a MR correlate of early neurological improvement”  In _Stroke_ 43.11 Am Heart Assoc, 2012, pp. 2986–2991
  * [49] BD Murphy et al.  “Identification of penumbra and infarct in acute ischemic stroke using computed tomography perfusion–derived blood flow and blood volume measurements”  In _Stroke_ 37.7 Am Heart Assoc, 2006, pp. 1771–1777  DOI: 10.1161/01.STR.0000227243.96808.53
  * [50] Luca Tomasetti et al.  “Machine Learning Algorithms Versus Thresholding to Segment Ischemic Regions in Patients With Acute Ischemic Stroke”  In _IEEE Journal of Biomedical and Health Informatics_ 26.2 IEEE, 2021, pp. 660–672
  * [51] Luca Tomasetti, Kjersti Engan, Mahdieh Khanmohammadi and Kathinka Dæhli Kurz  “CNN Based Segmentation of Infarcted Regions in Acute Cerebral Stroke Patients From Computed Tomography Perfusion Imaging”  In _Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics_ , 2020, pp. 1–8
  * [52] Luca Tomasetti et al.  “Multi-input segmentation of damaged brain in acute ischemic stroke patients using slow fusion with skip connection”  In _Proceedings of the Northern Lights Deep Learning Workshop_ 3, 2022
  * [53] Stephen M Smith  “Fast robust automated brain extraction”  In _Human brain mapping_ 17.3 Wiley Online Library, 2002, pp. 143–155
  * [54] Mohamed Najm et al.  “Automated brain extraction from head CT and CTA images using convex optimization with shape propagation”  In _Computer Methods and Programs in Biomedicine_ 176 Elsevier, 2019, pp. 1–8
  * [55] Nils Gessert, Marcel Bengs, Matthias Schlüter and Alexander Schlaefer  “Deep learning with 4D spatio-temporal data representations for OCT-based force estimation”  In _Medical image analysis_ 64 Elsevier, 2020, pp. 101730
  * [56] Marcel Bengs, Nils Gessert and Alexander Schlaefer  “4D spatio-temporal convolutional networks for object position estimation in OCT volumes”  In _Current directions in biomedical engineering_ 6.1 De Gruyter, 2020
  * [57] Andriy Myronenko et al.  “4D CNN for semantic segmentation of cardiac volumetric sequences”  In _International Workshop on Statistical Atlases and Computational Models of the Heart_ , 2019, pp. 72–80  Springer
  * [58] Nabila Abraham and Naimul Mefraz Khan  “A novel focal tversky loss function with improved attention u-net for lesion segmentation”  In _2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)_ , 2019, pp. 683–687  IEEE  DOI: 10.1109/ISBI.2019.8759329
  * [59] Fausto Milletari, Nassir Navab and Seyed-Ahmad Ahmadi  “V-net: Fully convolutional neural networks for volumetric medical image segmentation”  In _2016 Fourth International Conference on 3D Vision (3DV)_ , 2016, pp. 565–571  IEEE
  * [60] Shaojie Bai, J Zico Kolter and Vladlen Koltun  “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling”  In _arXiv preprint arXiv:1803.01271_ , 2018
  * [61] Yarin Gal and Zoubin Ghahramani  “Dropout as a bayesian approximation: Representing model uncertainty in deep learning”  In _international conference on machine learning_ , 2016, pp. 1050–1059  PMLR
  * [62] Ozan Oktay et al.  “Attention u-net: Learning where to look for the pancreas”  In _arXiv preprint arXiv:1804.03999_ , 2018
  * [63] Benjamin Graham  “Spatially-sparse convolutional neural networks”  In _arXiv preprint arXiv:1409.6070_ , 2014
  * [64] Sil C Van De Leemput et al.  “Multiclass brain tissue segmentation in 4D CT using convolutional neural networks”  In _IEEE Access_ 7 IEEE, 2019, pp. 51557–51569
  * [65] Diederik P Kingma and Jimmy Ba  “Adam: A method for stochastic optimization”  In _arXiv preprint arXiv:1412.6980_ , 2014
  * [66] T Birsan and Dan Tiba  “One hundred years since the introduction of the set distance by Dimitrie Pompeiu”  In _IFIP Conference on System Modeling and Optimization_ , 2005, pp. 35–39  Springer
  * [67] Rajiv Advani, Halvor Naess and Martin W Kurz  “The golden hour of acute ischemic stroke”  In _Scandinavian journal of trauma, resuscitation and emergency medicine_ 25.1 Springer, 2017, pp. 1–5
  * [68] Atte Meretoja et al.  “Stroke thrombolysis: save a minute, save a day”  In _Stroke_ 45.4 Am Heart Assoc, 2014, pp. 1053–1058
  * [69] Atte Meretoja et al.  “Endovascular therapy for ischemic stroke: save a minute—save a week”  In _Neurology_ 88.22 AAN Enterprises, 2017, pp. 2123–2127
  * [70] Satya P Singh et al.  “3D deep learning on medical images: a review”  In _Sensors_ 20.18 Multidisciplinary Digital Publishing Institute, 2020, pp. 5097
  * [71] Nikita Lakomkin et al.  “Prevalence of large vessel occlusion in patients presenting with acute ischemic stroke: a 10-year systematic review of the literature”  In _Journal of neurointerventional surgery_ 11.3 British Medical Journal Publishing Group, 2019, pp. 241–245
  * [72] Arsany Hakim et al.  “Predicting infarct core from computed tomography perfusion in acute ischemia with machine learning: Lessons from the ISLES challenge”  In _Stroke_ 52.7 Am Heart Assoc, 2021, pp. 2328–2337

|  Luca Tomasetti (M’2021) received the B.S. degree in computer science from
Università di Bologna, Bologna, Italy, in 2015 and the M.S. degree in computer
science from University of Stavanger (UiS), Stavanger, Norway, in 2019. He is
currently pursuing the Ph.D. degree in artificial intelligence at the Computer
Science Department of UiS. He is part of the project “In depth analysis of
perfusion computed tomography (CTP) in patients with acute ischemic stroke” at
UiS. The project is part of a twin project performed in close cooperation with
a PhD candidate in radiology. His research interest are in image processing
and machine learning with emphasis on ischemic stroke. He is also a member of
the Biomedical data analysis laboratory (BMDLab).
---|---
|  Kjersti Engan (M’1996, SM’2018) is a professor at the Electrical
Engineering and Computer Science department at the University of Stavanger
(UiS), Norway. She received the BE degree in electrical engineering from
Bergen University College in 1994 and the M.Sc. and Ph.D degrees in 1996 and
2000 respectively, in electrical engineering and information technology from
the UiS. She is the leader of the Biomedical data analysis lab - BMDLab - at
UiS. Her research interests are in signal and image processing and machine
learning with emphasis on medical applications and in dictionary learning for
sparse signal and image representation. She has a particular interest in AI
for newborn survival, stroke detection from CTP imaging and AI in
computational pathology. She is a senior member of IEEE. She has served as
Associate editor and Senior Area editor for IEEE Signal Processing Letters and
as a member of IEEE Image, Video, and Multidimensional Signal Processing
Technical Committee (IVMSP) and associate editor for SIAM Journal on Imaging
Sciences (SIIMS).
---|---
|  Liv Jorunn HØllesli (MD, PhD candidate) is at the end of her PhD candidacy
on the project “In depth analysis of perfusion computed tomography (CTP) in
patients with acute ischemic stroke” at the Electrical Engineering and
Computer Science Department at the University of Stavanger (UiS), Norway. The
project is part of a twin project performed in close cooperation with a PhD
student on computer science. Dr. Høllesli is also a senior consultant
radiologist at Stavanger University Hospital, Norway, holding the European
Diploma in Neuroradiology. She received the cand.med.-degree from the Faculty
of Medicine, University of Bergen, Norway, in 2012. Neuroradiology, with focus
on medical imaging and AI in ischemic stroke, is her primary research focus.
Høllesli is a member of the European Society of Neuroradiology, the Norwegian
Society of Neuroradiology and the Norwegian Medical association
---|---
|  Kathinka DÆhli Kurz (MD, PhD, Professor II) is a senior consultant
radiologist at Stavanger University Hospital, Norway, holding the European
Diploma in Neuroradiology. She also has a professor II position at the
Electrical Engineering and Computer Science Department at the University of
Stavanger (UiS), Norway. She received the cand.med.-degree from the Albert-
Ludwigs University of Freiburg, Germany, in 2000. She holds a PhD degree from
Germany (2004), and a PhD degree on “Dynamic MRI: An important tool in
problem-solving of breast abnormalities” from the University of Oslo, Norway
(2011). Her primary research interests are ischemic stroke, neuroradiology and
magnetic resonance imaging. Kurz is a member of the European Society of
Neuroradiology, the Norwegian Society of Neuroradiology and the Norwegian
Medical association.
---|---
|  Mahdieh Khanmohammadi (M’2013), an associate professor at the University of
Stavanger, has made significant contributions to the field of medical image
analysis. She was born and raised in Iran pursued her passion for science and
technology by studying bioelectrical engineering as her bachelor and continued
to master’s degree in the same field at the University of Tehran. Mahdieh
obtained her second master’s degree in computational engineering from
University of Halmstad in Sweden. Motivated by her master’s research, Mahdieh
pursued a Ph.D. program at the University of Copenhagen in Denmark. There, she
joined the image processing group at the Computer Science Department, working
on a project in collaboration with the neuroscientists at Aarhus University
and the Statistical and Mathematical Science Department at Aalborg University.
Her doctoral research involved analyzing the effect of behavioral stress on
the brain, utilizing microscopic images of rat brains. Following the
completion of her Ph.D., Mahdieh continued her academic journey as a
postdoctoral researcher at the University of Stavanger. In collaboration with
cardiologists at Stavanger University Hospital, she focused on analyzing
angiographic data of the heart to estimate blood velocity.
---|---