# Automatic Detection of Nuclear Spins at Arbitrary Magnetic Fields via
Signal-to-Image AI Model

B. Varona-Uriarte  Department of Physical Chemistry, University of the Basque
Country UPV/EHU, Apartado 644, 48080 Bilbao, Spain  EHU Quantum Center,
University of the Basque Country UPV/EHU, Leioa, Spain  C. Munuera-Javaloy
Department of Physical Chemistry, University of the Basque Country UPV/EHU,
Apartado 644, 48080 Bilbao, Spain  EHU Quantum Center, University of the
Basque Country UPV/EHU, Leioa, Spain  E. Terradillos  TECNALIA, Digital Unit
Quantum Technologies Group. The Basque Country Technology Park. Astondo Bidea,
Edificio 700, 48160 Derio, Spain  A. Alvarez-Gila  TECNALIA, Digital Unit
Quantum Technologies Group. The Basque Country Technology Park. Astondo Bidea,
Edificio 700, 48160 Derio, Spain  E. Garrote  TECNALIA, Digital Unit Quantum
Technologies Group. The Basque Country Technology Park. Astondo Bidea,
Edificio 700, 48160 Derio, Spain  Department of Automatic Control and Systems
Engineering, University of the Basque Country UPV/EHU, 48013 Bilbao, Spain  J.
Casanova  Department of Physical Chemistry, University of the Basque Country
UPV/EHU, Apartado 644, 48080 Bilbao, Spain  EHU Quantum Center, University of
the Basque Country UPV/EHU, Leioa, Spain  IKERBASQUE, Basque Foundation for
Science, Plaza Euskadi 5, 48009 Bilbao, Spain

###### Abstract

Quantum sensors leverage matter’s quantum properties to enable measurements
with unprecedented spatial and spectral resolution. Among these sensors, those
utilizing nitrogen-vacancy (NV) centers in diamond offer the distinct
advantage of operating at room temperature. Nevertheless, signals received
from NV centers are often complex, making interpretation challenging. This is
especially relevant in low magnetic field scenarios, where standard
approximations for modeling the system fail. Additionally, NV signals feature
a prominent noise component. In this work, we present a signal-to-image deep
learning model capable to automatically infer the number of nuclear spins
surrounding an NV sensor and the hyperfine couplings between the sensor and
the nuclear spins. Our model can be trained to operate effectively across
various magnetic field scenarios, requires no prior knowledge of the involved
nuclei, and is designed to handle noisy signals, leading to fast
characterization of nuclear environments in real experimental conditions. With
detailed numerical simulations, we test the performance of our model in
scenarios involving varying numbers of nuclei, achieving an average error of
less than $2\ \rm{kHz}$ in the estimated hyperfine constants.

_Introduction.–_ Fast and precise characterization of quantum registers is a
central issue in the realm of quantum technologies, with vast applications in
communication, computing, sensing, and simulation [1]. In particular, the
efficient analysis of systems that comprise electron-like defects and nuclear
spins in solid-state materials is pivotal for advancing quantum networks and
quantum information processing [2, 3]. In this context, deep learning-based
approaches offer valuable solutions for addressing the challenges associated
with quantum characterization [4, 5, 6].

From a more technological perspective, nuclear-spin detection has experienced
significant advancements with the use of NV centers in diamond [7, 8]. NVs
exhibit long coherence times [9] at room temperature [10, 11], which make them
well-suited for a wide range of applications in the field of biological
analysis [12, 13, 14]. Additionally, NV-based quantum sensors are easy to
initialize and readout by optical means [15, 16, 17], while their hyperfine
levels can be coherently manipulated using microwave (MW) radiation [18, 19].
Regarding the NV environment, naturally $1.1\ \%$ of carbon nuclei are
${}^{13}\rm{C}$ featuring a spin-$\frac{1}{2}$ [20]. Consequently, NVs couple
to each ${}^{13}\rm{C}$ in the diamond lattice through a hyperfine vector
$\vec{A}$ typically underlying dipolar interactions. Hence, rapid and precise
characterization of each hyperfine vector would significantly contribute to
asses the potential of specific quantum nodes, consisting of NVs and nuclei,
for distinct quantum information processing tasks.

In this Letter, we present the SALI (Signal-to-image ArtificiaL Intelligence)
model, able to characterize quantum nodes by identifying the number of
involved nuclear spins and accurately estimating the hyperfine parameters of
each NV-nucleus interaction. SALI comprises a 1D $\rightarrow$ 2D
Convolutional Neural Network (CNN) module for processing 1D string data of NV
measurements into a 2D image output, along with an image post-processing
module. It is a compact model that offers automatic characterization (meaning
it operates as a black box, without requiring human intervention) of nuclei in
proximity to a probe NV, achieving the processing task within milliseconds.
Notably, our model: (i) Exhibits high accuracy in predicting the value of
hyperfine vectors over a wide range of values, (ii) Effectively handles noisy
signals commonly encountered in experimental scenarios, (iii) Does not require
prior knowledge of the number of nuclei in each node, and more importantly,
(iv) Performs well in low-field conditions, where conventional approximations
break down, resulting in intricate and challenging signals. Through detailed
numerical simulations, we evaluate the performance of our model in nodes
containing up to 20 nuclei, considering both high and low magnetic field
scenarios.

_The system.–_ We consider a quantum node consisting of an NV and $n$
${}^{13}\rm{C}$ nearby nuclear spins with Larmor frequency
$\omega_{\rm{L}}=\gamma_{\rm{n}}B_{z}$, such that
$\gamma_{\rm{n}}=(2\pi)\times 10.705$ MHz/T, while the magnetic field $B_{z}$
is aligned with the NV axis ($\hat{z}$). The Hamiltonian that describes this
system is

$H=\sum_{j=1}^{n}\omega_{j}\
\hat{\omega}_{j}\cdot\vec{I}_{j}+\frac{f(t)}{2}\sigma_{z}\sum_{j=1}^{n}\vec{A}_{j}\cdot\vec{I}_{j},$
(1)

Figure 1: Schematic representation of the SALI model. The neural network takes
two input signals ($P_{x}$) obtained from CPMG sequences of different
duration. In block (i) of the neural network, each input signal is
independently processed through two 1-dimensional convolutional blocks. Each
block comprises two convolutional layers with $F$ filters of kernel size 3,
followed by a batch normalization (BN) layer, a ReLU activation function, and
a max-pooling layer with a window size of 2. The flattened outputs from these
convolutional blocks are then concatenated and connected to block (ii), a
fully connected block. The last dense layer in this block is reshaped into a
rectangular image. Within block (iii), the second convolutional layer is a
transposed convolutional layer with a stride of 2, effectively doubling the
image’s width and height. The 2-dimensional convolutional layers utilize $F$
filters with a kernel size of (3, 3). Block (iii) is subsequently connected to
the output convolutional layer of the neural network, which is followed by an
image post-processing module to extract the number of nuclei ($n$) within the
sample and the corresponding coupling constant pairs
$(A_{j}^{z},A_{j}^{\bot})$. See in-depth explanation in SM [21].

where $\omega_{j}\ \hat{\omega}_{j}=\gamma_{\rm{n}}B_{z}\
\hat{z}+\frac{1}{2}\vec{A}_{j}$, $\vec{A}_{j}=(A_{j}^{z},A_{j}^{\bot})$ is the
hyperfine vector joining the NV with the $j$th nucleus with spin operator
$\vec{I}_{j}$, and $f(t)=\pm 1$ is the modulation function that appears as a
consequence of the introduced MW driving (for more details see Supplemental
Material (SM) [21]). In particular, we consider trains of $\pi$-pulses over
the NV (which is initialized to the $|+\rangle$ state, such that
$\sigma_{x}|+\rangle=|+\rangle$) according to the CPMG sequence [22, 23],
after which the NV is measured. Repeating this process $\rm{N}_{\rm{m}}$
times, one estimates the survival probability ($P_{x}$) of the initial state
$|+\rangle$. In the ideal scenario of infinite number of measurements (note
our numerical simulations consider a finite number of measurements and
decoherence effects) $P_{x}$ reads

$P_{x}=\frac{1}{2}\cdot\left(1+\displaystyle\prod_{j=1}^{n}M_{j}\right),$ (2)

where

$M_{j}=1-m_{j,x}^{2}\frac{(1-\cos{\alpha_{j}})(1-\cos{\beta})}{1+\cos{\alpha_{j}}\cos{\beta}-m_{j,z}\sin{\alpha_{j}}\sin{\beta}}\sin{\frac{N\phi_{j}}{2}}^{2},$
(3)

$\cos{\phi_{j}}=\cos{\alpha_{j}}\cos{\beta}-m_{j,z}\sin{\alpha_{j}}\sin{\beta},$
(4)

with $m_{j,z}=\frac{(A_{j}^{z}+\omega_{L})}{\tilde{\omega_{j}}}$,
$m_{j,x}=\frac{A_{j}^{\bot}}{\tilde{\omega_{j}}}$,
$\tilde{\omega}=\sqrt{(A_{j}^{z}+\omega_{L})^{2}+{A_{j}^{\bot}}^{2}}$,
$\alpha_{j}=\tilde{\omega_{j}}\tau$, $\beta=\omega_{L}\tau$, and $\tau$ is
half the interpulse spacing of a CPMG sequence (see full derivation in
Supplemental Material from reference [1]). In a scenario such that
$\omega_{\rm{L}}\gg A_{j}^{z},A_{j}^{\bot}$ (namely, at high magnetic field,
$B_{z}=0.056\rm{T}$ in our case) $P_{x}$ exhibits clear resonance peaks at
$\tau=\frac{k\pi}{2\omega_{j}}$. In this context, techniques based on
classical algorithms [25], as well as deep learning models [4, 26, 27], are
used to find $A_{j}^{z}$ and $A_{j}^{\bot}$. However, in the low-field regime,
where the condition $\omega_{\rm{L}}\gg A_{j}^{z},A_{j}^{\bot}$ does not hold
($B_{z}=0.0056\rm{T}$ in our particular case), resonance peaks cannot be
observed [21]. More specifically, in this regime $P_{x}$ shows an intricate
behavior which makes previously mentioned techniques for system
characterization challenging.

Now we introduce our SALI model designed to effectively process complex
signals across diverse magnetic field scenarios, leading to
$(A_{j}^{z},A_{j}^{\bot})$ as output. Our model showcases robust performance
in handling noisy signals commonly encountered in experimental scenarios and
operates seamlessly without requiring any prior information about the number
of nuclei involved in the node.

Figure 2: (a) True output image of the low-field neural network. 10 nuclei
with random coupling constants in the ranges $A^{z}\in[-100,\ 100]\ \rm{kHz}$
and $A^{\bot}\in[2,\ 102]\ \rm{kHz}$ are represented. Each nucleus appears as
a $5\times 5$ pixel region in the image, as depicted in the highlighted
circular area. The pixel values in this region are chosen to match a Gaussian
distribution centered in $(A_{j}^{z},A_{j}^{\bot})$. (b) Output image
predicted by the neural network. (c) Output image after the image post-
processing. The white boxes represent predicted nuclei. (d) Evaluation of the
model performance. The predicted nuclei (white boxes) are compared to the true
nuclei (green boxes). In this example, we observe 9 TPs (true positives, i.e.
correctly detected nuclei), 1 FP (false positive, i.e. predicted non-existent
nucleus), and 1 FN (false negative, i.e. non-predicted nucleus). In the
highlighted circular area we examine a specific TP nucleus. The $5\times 5$
green box bounds the nucleus in the true output, while the white box bounds
the nucleus in the predicted output.

_The SALI model.–_ A scheme of SALI is given in Fig. 1 (see caption for
details on the architecture of the model). The 1D $\rightarrow$ 2D CNN module
takes two input signals ($P_{x}$) coming from CPMG sequences with different
number of pulses ($\rm{N}=32$ and $\rm{N}=256$ in our specific example). In
this way, each sequence presents different evolution times, allowing the
network to infer both weak and strongly coupled nuclei. The architecture of
the neural network is as follows: (i) Two separate 1D CNN blocks analyze the
inputs. After processing the signals, the outputs of these blocks are
flattened and concatenated into a single array. (ii) A fully connected block
is introduced between the 1D CNN block and the next 2D CNN block, serving as
an intermediary between these two blocks, and additionally, allowing for the
adaptation of the final output image size. The outcome of the fully connected
block is reshaped into a 2-dimensional array, treated as an image to exploit
the spatial relations among adjacent pixels. (iii) In the 2D CNN block, the
reshaped image is processed. Finally, this block connects to the output layer
of the neural network (NN OUTPUT in Fig. 1), which is a convolutional layer
with sigmoid activation function that encodes the target parameters
$(A_{j}^{z},A_{j}^{\bot})$ in a 2-dimensional image.

For the training, validation, and testing of the neural network, we generated
two distinct datasets, one for the high magnetic field scenario and the other
for the low magnetic field scenario, each comprising 3,600,000 samples. Each
sample within these datasets contains a random number of nuclei ranging from 1
to 20, with each nucleus having random values of $A^{z}$ and $A^{\bot}$ in the
ranges $A^{z}\in[-100,\ 100]\ \rm{kHz}$ and $A^{\bot}\in[2,\ 102]\ \rm{kHz}$.
Note that Eqs. (2-4) dictate that $P_{x}$ is symmetric with respect to the
change $A^{\bot}\rightarrow-A^{\bot}$. Consequently, we only consider positive
values for $A^{\bot}$. The input data strings ($P_{x}$) are generated with
$\rm{N}=32$ $\pi$-pulses, with $\tau$ varying in the range $\tau_{32}\in[6,\
50]\ \rm{\mu s}$, and $\rm{N}=256$ $\pi$-pulses, with $\tau$ in the range
$\tau_{256}\in[10,\ 40]\ \rm{\mu s}$. Each $P_{x}$ contains
$\rm{N}_{\rm{p}}=1000$ points, resulting in resolutions of $\Delta t_{32}=44\
\rm{ns}$ and $\Delta t_{256}=30\ \rm{ns}$. Thus, the employed CPMG sequences
would last some milliseconds. We consider experimental conditions akin to
those in [8]. This is, we model potential decoherence effects over the NV
sensor by adding an exponential factor $e^{-\tau/T_{2}}$ to $P_{x}$ with
$T_{2}=200\ \rm{\mu s}$. Furthermore, we consider shot-noise by computing each
average value in $P_{x}$ after collecting $\rm{N}_{\rm{m}}=1000$ measurements.

To supervise the training and validation stages, as well as for evaluation in
the testing, nuclei are depicted in the true output image of the neural
network as exemplified in Fig. 2(a). This portrayal involves a fuzzy logic
approximation through the use of Gaussians, reminiscent of representing a
point in an image. As illustrated in Fig. 2(a), each nucleus is depicted
within a $5\times 5$ pixel region centered around the nearest pixel to the
true values $(A_{j}^{z},A_{j}^{\bot})$.

After the training process, the predicted output of the neural network
exhibits distinct clusters of pixels, ideally resembling Gaussian functions
(see Fig. 2(b)). The next step is to analyze these clusters through the image
post-processing module. This process begins with the application of erosion
and dilation techniques, which are employed to smooth the image. Next, we
perform a thresholding of the smoothed image and group the clusters of
adjacent pixels with a connectivity routine. Finally, an area filter is
applied to determine which predicted clusters qualify as nuclei (see white
boxes in Fig. 2(c)). The result is the prediction of the number of nuclei $n$
and the corresponding coupling constant pairs $(A_{j}^{z},A_{j}^{\bot})$,
which are determined by the centroids of these clusters.

In summary, our SALI model operates in two distinct phases: the 1D
$\rightarrow$ 2D CNN module and the image post-processing module. Within the
neural network module, the 1D CNN block extracts valuable information from the
input signals, enabling the representation of hyperfine couplings in a
2-dimensional image through the 2D CNN block. This resulting image serves as a
blank canvas on which the neural network paints the presence of an initially
unspecified number of nuclear spins and estimates their corresponding
hyperfine coupling constants. These estimates are subsequently numerically
derived through the image post-processing module. Technical details regarding
the model are in the SM [21], while the SALI code can be distributed upon
request to the authors.

Figure 3: (a) Precision ($P$) and recall ($R$) of the model with respect to
the number of nuclei present in the true samples. (b) Mean Absolute Error
(MAE) between the predicted and the true values of the coupling constants
$A^{z}$ (green) and $A^{\bot}$ (red). This metric is calculated exclusively
for TPs. (c) In yellow (blue), the MAE between the original and predicted
input signals, corresponding to $\rm{N}=32$ ($\rm{N}=256$) pulses. The solid
(dashed) lines in (a)-(c) correspond to the high (low) magnetic field
scenario. (d)-(g) Signal comparisons from samples containing 16 nuclei. In
black, the signals calculated with true nuclei. In yellow (blue), the
$\rm{N}=32$ ($\rm{N}=256$) signals obtained with SALI’s predicted nuclei. (d)
and (e) correspond to the low-field scenario (14 nuclei are detected), and (f)
and (g) to high field (all 16 nuclei are detected). More details in [21].

_Quantifying the model performance and results.–_ The values predicted by the
model are then compared to the true values using the following procedure.
Initially, all nuclei in an image sample are enclosed within bounding boxes:
the nuclei in the true sample (“true nuclei”) are bounded in $5\times 5$ green
boxes and the nuclei in the predicted sample (“predicted nuclei”) are bounded
through the image post-processing module in white boxes, as seen in Fig. 2(d).
For each predicted nucleus box, the Intersection over Union (IoU) with each
true nucleus box is calculated, defined as the area of overlap over the area
of union, ideally being 1; the true nucleus box with the largest IoU is
considered the detected true nucleus.

We employ two standard classification metrics to evaluate the prediction of
the number of nuclei: precision, $P=\frac{\rm{TP}}{\rm{TP}+\rm{FP}}$, and
recall, $R=\frac{\rm{TP}}{\rm{TP}+\rm{FN}}$. These metrics are computed based
on every nucleus within each sample, where true positives (TP) are correctly
detected nuclei, false negatives (FN) are undetected nuclei, and false
positives (FP) are non-existent nuclei incorrectly detected. High precision
(note $P\in[1,0]$) indicates the network’s capability to accurately identify
existing nuclei without incorporating false ones, while high recall
($R\in[1,0]$) signifies that the network detects a large number of existing
nuclei. In Fig. 3(a), we present the average of these two metrics calculated
for the test subset (15% of the entire dataset split before training). We have
evaluated the performance of coupling constants $(A^{z},A^{\bot})$ estimation
for the detected nuclei (TPs) using Mean Absolute Error (MAE) between
predicted and true values, shown in Fig. 3(b), where the MAE remains below $2\
\rm{kHz}$ in all cases.

Finally, we assess the similarity between the original input signals and the
ones generated with the complete set of predicted $(A_{j}^{z},A_{j}^{\bot})$
values by computing the MAE between the original and predicted signals for
$\rm{N}=32$ and $\rm{N}=256$ (refer to Fig. 3(c)). Notably, although precision
and recall decrease as the number of nuclei increases, the predicted signals
closely resemble the original signals –as indicated by the low MAEs in Fig.
3(c). This conveys the efficacy of SALI in capturing the essential traits of
quantum nodes, see Figs. 3(d)-(g).

_Conclusions.–_ Employing deep learning models and computer vision algorithms
offers a key advantage: rapid and automated detection. Our SALI model achieves
large accuracy in predicting hyperfine vectors, effectively handles noisy
signals common in experiments, does not require prior knowledge about the
number of nuclei in each node, and operates even at low-field conditions.
Hence, it serves as a valuable tool for scientists engaged in solid-state
platforms, particularly in quantum sensing and quantum information processing.
While our investigation has successfully demonstrated SALI’s ability to
accurately reproduce input signals, thereby confirming its efficacy, there is
potential for further enhancement. One approach to achieve this is by
introducing additional signals during the training stage (note, for the sake
of simplicity we consider two inputs) obtained with diverse sequences [28,
29]. This approach seeks to reinforce the detection process in scenarios
characterized by a considerable number of nuclei, particularly in the low-
field regime.

_Acknowledgements.–_ C. M.-J. acknowledges the predoctoral MICINN Grant No.
PRE2019-088519. J. C. acknowledges the Ramón y Cajal (RYC2018-025197-I)
research fellowship, the financial support from Spanish Government via the
Nanoscale NMR and complex systems (PID2021-126694NB-C21) project, the Basque
Government grant IT1470-22. J.C, E.G., and E.T. ackowledge the ELKARTEK
project Dispositivos en Tecnologías Cuánticas (KK-2022/00062).

## References

  * [1] V. Gebhart, R. Santagati, A. A. Gentile, E. M. Gauger, D. Craig, N. Ares, L. Banchi, F. Marquardt, L. Pezze’, and C. Bonato, Learning Quantum Systems, Nat. Rev. Phys. 5, 141 (2023).
  * [2] D. D. Awschalom, R. Hanson, J. Wrachtrup, and B. B. Zhou, Quantum technologies with optically interfaced solid-state spins, Nat. Photonics 12, 516 (2018).
  * [3] F. A. Zwanenburg, A. S. Dzurak, A. Morello, M. Y. Simmons, L. C. Hollenberg, G. Klimeck, S. Rogge, S. N. Coppersmith, and M. A. Eriksson, Silicon quantum electronics, Reviews of Modern Physics 85, 961 (2013).
  * [4] K. Jung, M. H. Abobeih, J. Yun, H. Kim, H. Oh, A. Henry, T. H. Taminiau, and D. Kim, Deep learning enhanced individual nuclear-spin detection, Npj Quantum Inf. 7, 41 (2021).
  * [5] Y. Chen, Y. Ban, R. He, J. M. Cui, Y. F. Huang, C. F. Li, G. C. Guo, and J. Casanova, A neural network assisted 171Yb+ quantum magnetometer, Npj Quantum Inf. 8, 152 (2022).
  * [6] Y. Ban, J. Echanobe, Y. Ding, R. Puebla, and J. Casanova, Neural-network-based parameter estimation for quantum detection, Quantum Science and Technology 6, 045012 (2021).
  * [7] C. E. Bradley, J. Randall, M. H. Abobeih, R. C. Berrevoets, M. J. Degen, M. A. Bakker, M. Markham, D. J. Twitchen, and T. H. Taminiau, A Ten-Qubit Solid-State Spin Register with Quantum Memory up to One Minute, Phys. Rev. X 9, 031045 (2019).
  * [8] V. Vorobyov, J. Javadzade, M. Joliffe, F. Kaiser, and J. Wrachtrup, Addressing Single Nuclear Spins Quantum Memories by a Central Electron Spin, Appl. Magn. Reson. 53, 1317 (2022).
  * [9] M. H. Abobeih, J. Cramer, M. A. Bakker, N. Kalb, M. Markham, D. J. Twitchen, and T. H. Taminiau, One-second coherence for a single electron spin coupled to a multi-qubit nuclear-spin environment, Nat. Commun. 9, 2552 (2018).
  * [10] S. Nomura, H. Watanabe, and S. Kashiwaya, Imaging of high-frequency electromagnetic field by multipulse sensing using nitrogen vacancy centers in diamond, arXiv:2307.02089 (2023).
  * [11] K. Ohashi, T. Rosskopf, H. Watanabe, M. Loretz, Y. Tao, R. Hauert, S. Tomizawa, T. Ishikawa, J. Ishi-Hayase, S. Shikata, C. L. Degen, and K. M. Itoh, Negatively Charged Nitrogen-Vacancy Centers in a 5 nm Thin 12C Diamond Film, Nano Lett. 13, 4733 (2013).
  * [12] Y. Wu, F. Jelezko, M. B. Plenio, and T. Weil, Diamond Quantum Devices in Biology, Angew. Chem. Int. Ed. 55, 6586 (2016).
  * [13] R. D. Allert, K. D. Briegel, and D. B. Bucher, Advances in nano- and microscale NMR spectroscopy using diamond quantum sensors, Chem. Commun. 5, 8165 (2022).
  * [14] T. Zhang, G. Pramanik, K. Zhang, M. Gulka, L. Wang, J. Jing, F. Xu, Z. Li, Q. Wei, P. Cigler, and Z. Chu, Toward Quantitative Bio-sensing with Nitrogen-Vacancy Center in Diamond, ACS Sens. 6, 2077 (2021).
  * [15] F. Jelezko, T. Gaebel, I. Popa, A. Gruber, and J. Wrachtrup, Observation of Coherent Oscillations in a Single Electron Spin Phys. Rev. Lett. 92, 076401 (2004).
  * [16] I. Aharonovich, C. Santori, B. A. Fairchild, J. Orwa, K. Ganesan, K. M. C. Fu, R. G. Beausoleil, A. D. Greentree, and S. Prawer, Producing optimized ensembles of nitrogen-vacancy color centers for quantum information applications, J. Appl. Phys. 106, (2009).
  * [17] V. V. Dobrovitski, G. D. Fuchs, A. L. Falk, C. Santori, and D. D. Awschalom, Quantum Control over Single Spins in Diamond, Annu. Rev. Condens. Matter Phys. 4, 23 (2013).
  * [18] P. Tamarat, N. B. Manson, J. P. Harrison, R. L. McMurtrie, A. Nizovtsev, C. Santori, R. G. Beausoleil, P. Neumann, T. Gaebel, F. Jelezko, P. Hemmer, and J. Wrachtrup, Spin-flip and spin-conserving optical transitions of the nitrogen-vacancy centre in diamond, New J. Phys. 10, 045004 (2008).
  * [19] P. Neumann, N. Mizuochi, F. Rempp, P. Hemmer, H. Watanabe, S. Yamasaki, V. Jacques, T. Gaebel, F. Jelezko, and J. Wrachtrup, Multipartite Entanglement Among Single Spins in Diamond, Science 320, 1326 (2008).
  * [20] A. O. Nier, A Redetermination of the Relative Abundances of the Isotopes of Carbon, Nitrogen, Oxygen, Argon, and Potassium, Phys. Rev. 77, 789 (1950).
  * [21] More detailed information can be found in the Supplemental Material.
  * [22] H. Y. Carr and E. M. Purcell, Effects of Diffusion on Free Precession in Nuclear Magnetic Resonance Experiments, Phys. Rev. 94, 630 (1954).
  * [23] S. Meiboom and D. Gill, Modified Spin-Echo Method for Measuring Nuclear Relaxation Times, Rev. Sci. Instrum. 29, 688 (1958).
  * [24] T. H. Taminiau, J. J. T. Wagenaar, T. Van der Sar, F. Jelezko, V. V. Dobrovitski, and R. Hanson, Detection and Control of Individual Nuclear Spins Using a Weakly Coupled Electron Spin, Phys. Rev. Lett. 109, 137602 (2012).
  * [25] H. Oh, J. Yun, M. H. Abobeih, K. H. Jung, K. Kim, T. H. Taminiau, and D. Kim, Algorithmic decomposition for efficient multiple nuclear spin detection in diamond, Sci. Rep. 10, 14884 (2020).
  * [26] A. Laraoui, F. Dolde, C. Burk, F. Reinhard, J. Wrachtrup, and C. A. Meriles, High-resolution correlation spectroscopy of 13C spins near a nitrogen-vacancy centre in diamond, Nat. Commun. 4, 1651 (2013).
  * [27] T. C. Farrar and E. D. Becker, Pulse and Fourier Transform NMR: Introduction to Theory and Methods, Elsevier (2012).
  * [28] J. Casanova, Z. Y. Wang, J. F. Haase, and M. B. Plenio, Robust Dynamical Decoupling Sequences for Individual Nuclear Spin Addressing, Phys. Rev. A 92, 042304 (2015).
  * [29] J. F. Haase, Z. Y. Wang, J. Casanova, and M. B. Plenio, Soft Quantum Control for Highly Selective Interactions among Joint Quantum Systems, Phys. Rev. Lett. 121, 050402 (2018).

Supplemental Material:
Automatic Detection of Nuclear Spins at Arbitrary Magnetic Fields via Signal-
to-Image AI Model

##  I Description of the system

We consider a system composed of an NV and $n$ ${}^{13}\rm{C}$ nearby nuclear
spins with Larmor frequency $\omega_{\rm{L}}=\gamma_{\rm{n}}B_{z}$, such that
$\gamma_{\rm{n}}=(2\pi)\times 10.705$ MHz/T, while the magnetic field $B_{z}$
is aligned with the NV axis ($\hat{z}$). The Hamiltonian that describes this
system reads as

$H_{I}=\sum_{j=1}^{n}\omega_{j}\
\hat{\omega}_{j}\cdot\vec{I}_{j}+\frac{1}{2}\sigma_{z}\sum_{j=1}^{n}\vec{A}_{j}\cdot\vec{I}_{j}+\frac{\Omega}{2}\sigma_{\phi},$
(S1)

where $\omega_{j}\ \hat{\omega}_{j}=\gamma_{\rm{n}}B_{z}\
\hat{z}+\frac{1}{2}\vec{A}_{j}$ are the effective Larmor frecuencies, and
$\vec{A}_{j}=(A_{j}^{z},A_{j}^{\bot})$ are the hyperfine coupling parameters.

The pulses of the CPMG sequence are applied through a microwave driving
represented as the final term in the Hamiltonian from Eq. (S1). Specifically,
the CPMG sequence involves the application of $\rm{N}$ $\pi$-pulses with an
interpulse delay of $2\tau$. In a rotating frame with respect to
$H_{0}=\frac{\Omega}{2}\sigma_{\phi}$ and applying this pulse sequence, one
obtains the following Hamiltonian:

$H=\sum_{j=1}^{n}\omega_{j}\
\hat{\omega}_{j}\cdot\vec{I}_{j}+\frac{f(t)}{2}\sigma_{z}\sum_{j=1}^{n}\vec{A}_{j}\cdot\vec{I}_{j},$
(S2)

where the modulation function $f(t)$ is a step function that alternates
between $+1$ and $-1$ each time a $\pi$-pulse is applied, as it can be seen in
Fig. S1(a).

By initializing the NV sensor in an eigenstate of $\sigma_{x}$ and applying
the aforementioned pulse sequence (i.e. CPMG), one can measure the final state
of the sensor ($\ket{0}_{x}$ or $\ket{1}_{x}$). Repeating this process
$\rm{N}_{m}$ times, one estimates the probability $P_{x}$ of the state of the
sensor being preserved. This probability can be computed for various values of
$\tau$ (half the interpulse delay), generating a signal of $P_{x}$ as a
function of $\tau$.

Figure S1: (a) Top panel, CPMG pulse sequence. Initially, a $\pi/2$-pulse is
applied to rotate the NV electron spin to the $XY$ plane. Subsequently,
$\rm{N}$ $\pi$-pulses are applied, and finally, another $\pi/2$-pulse is
applied before the measurement. Bottom panel, $f(t)$ modulation function
resulting from the application of the $\pi$-pulses in the CPMG sequence. (b)
and (c) $P_{x}$ calculated at $B_{z}=0.056\ \rm{T}$ and $B_{z}=0.0056\
\rm{T}$, respectively. Each sequence contains $\rm{N}=32$ $\pi$-pulses, and
$P_{x}$ is sampled $\rm{N}_{p}=1000$ times in the range $\tau\in[6,\ 50]\
\rm{\mu s}$. In this example, the node contains the NV and 3 nuclei with
$(3,75)\ \rm{kHz}$, $(-45,42)\ \rm{kHz}$, and $(23,4)\ \rm{kHz}$.

In Fig. S1(b)-(c), one can observe the $P_{x}$ performance for distinct values
of the magnetic field $B_{z}$. In particular, in Fig. S1(b)
($B_{z}=0.056\rm{T}$) the signal exhibits clear resonance peaks at specific
values of $\tau=\frac{k\pi}{2\omega_{j}}$, for $k$ odd. Fig. S1(c) shows the
behavior of $P_{x}$ in a low-field regime ($B_{z}=0.0056\rm{T}$), where these
resonances cannot be seen anymore (see mathematical derivation in Supplemental
Material from reference [1]).

##  II SALI MODEL

###  II.1 Dataset

We assessed the performance of the proposed SALI model in two scenarios: one
with an NV sensor and up to 20 ${}^{13}\rm{C}$ nuclei at high magnetic field
($B_{z}=0.056\ \rm{T}$), and the other with an NV sensor and up to 20
${}^{13}\rm{C}$ nuclei at low magnetic field ($B_{z}=0.0056\ \rm{T}$). The
samples were generated based on Eqs. (2-4) in the main text, with 3,600,000
samples for each case. All samples considered a random number of nuclei (up to
20) and random values of $A^{z}$ and $A^{\bot}$ within the ranges
$A^{z}\in[-100,\ 100]\ \rm{kHz}$ and $A^{\bot}\in[2,\ 102]\ \rm{kHz}$. Each
dataset was divided into training (70%), validation (15%), and test (15%)
subsets before network training to prevent bias. Additionally, all three
subsets (training, validation, and test) were normalized using the mean value
and variance of the training set. The normalization process is expressed as

$P_{x}\rightarrow\frac{P_{x}-\langle
P_{x}\rangle}{\sqrt{\rm{Var}(P_{x})+\epsilon}},$ (S3)

where $\epsilon=0.001$ is added tto the denominator to avoid divisions by
zero.

The outputs of the SALI model are the number of nuclei $n$ and the set of
coupling parameters $(A_{j}^{z},A_{j}^{\bot})$ between the NV sensor and the
nuclear spins. However, it is essential to define an intermediate output for
the CNN to supervise the training. Thus, the true output of the network is a
2-dimensional image where nuclei are depicted as Gaussian functions, denoted
as $g(x,y)=e^{-\frac{(x-x_{0})^{2}+(y-y_{0})^{2}}{2}}$ (see Fig S2(a)). Along
the $x$ axis, the perpendicular component of the coupling constant $A^{\bot}$
is represented, while the $y$ axis represents the parallel component $A^{z}$.
To portray each nucleus, the exact values of $A^{z}$ and $A^{\bot}$ are taken
as the peak positions of the Gaussian function ($x_{0}$ and $y_{0}$). The
value of the Gaussian at the closest pixel to the peak is used to represent
the nucleus, as well as the nearest neighbors (8 pixels) and next-nearest
neighbors (16 pixels). As a result, each nucleus is represented by a $5\times
5$ pixel region in the image. The neural network’s output is a $200\times 100\
\rm{kHz}$ grid, sized $(200+4)\times(100+4)$, resulting in pixel sizes of
$\Delta A=1.005\ \rm{kHz}$ and $\Delta B=1.01\ \rm{kHz}$. The “effective”
output is sized $200\times 100$, while the 4 extra pixels correspond to a
2-pixel added border to ensure that we can accurately represent nuclei
positioned right at the edge of the image, requiring up to 2 additional pixels
beyond the image boundary to represent the nearest and next-nearest neighbors.
Additionally, we have imposed an upper bound of 1 on pixel values. This
constraint addresses situations where two or more nuclei with very similar
coupling constants can cause their Gaussian functions to combine and exceed a
value of 1 within the same pixel.

This output representation of the neural network allows for the inclusion of
an arbitrary number of nuclei and their associated parameters. In terms of the
training efficiency of the deep learning model, this representation provides a
rich supervisory signal, with each pixel in the density map contributing to
the optimization gradients. Additionally, its versatility makes it suitable
for various deep learning architectures.

###  II.2 Model

The SALI model comprises two modules: a deep learning network followed by an
image processing routine. The model takes two inputs corresponding to signals
from CPMG sequences with $\rm{N}=32$ and a $\rm{N}=256$ $\pi$-pulses. The
output consists of the set of $(A_{j}^{z},A_{j}^{\bot})$ associated with each
of the nuclei predicted by the model. The overall architecture is depicted in
Fig. 1 in the main text, featuring a 1D $\rightarrow$ 2D Convolutional Neural
Network (CNN) and an image post-processing module.

####  II.2.1 1D $\rightarrow$ 2D Convolutional Neural Network (CNN)

The network architecture comprises 37 layers, organized into three broad
blocks:

  * •

1D CNN block: This block consists of two parallel branches designed to
independently process each input signal. The input signals undergo two
1-dimensional convolutional layers with 16 and 32 filters, each with a kernel
size of 3. Subsequently, these layers are followed by batch normalization, a
ReLU layer, and a 1D max-pooling layer that reduces the size of the preceding
layer by half. After these layers, a dropout layer with a rate of 0.2 is
introduced to mitigate overfitting. The outputs from these two parallel 1D CNN
blocks are flattened and then merged into a single array.

  * •

Fully connected block: The initial dense layer in this block has a size of
$(204/2\cdot 104/2)/2$, serving as the neural network’s bottleneck. The
subsequent layer is twice the size of the preceding dense layer ($204/2\cdot
104/2$). Both dense layers are followed by a batch normalization layer prior
to the ReLU activation function. A dropout layer with a rate of 0.2 is
incorporated between the dense layers. The final dense layer is then reshaped
into a $204/2\times 104/2$ image. This block acts as a transition block
between the 1D and 2D CNN blocks.

  * •

2D CNN block: This convolutional segment includes a 2-dimensional
convolutional layer with 32 filters and a $(3,3)$ kernel size, followed by a
2-dimensional transposed convolutional layer with 16 filters of the same
kernel size. Notably, this last layer employs a stride of 2, effectively
doubling the width and height of the input image to the layer. Batch
normalization layers are placed between the convolutional layers and the ReLU
activation functions, and a dropout layer with a 0.2 rate is added between the
convolutional layers. The output of the neural network is a 2D convolutional
layer with 1 filter, a $(3,3)$ kernel size, and a sigmoid activation function
that scales each pixel to fall within the range of 0 to 1.

####  II.2.2 Image post-processing

The image post-processing procedure begins with the application of erosion and
dilation filters to smooth the image and diminish spurious values in the
predicted network outputs (refer to Fig S2(b)). A threshold is then applied to
the smoothed image for refinement. Next, adjacent pixels are grouped by
connectivity to create clustered objects, followed by the application of an
area filter to remove clusters that are deemed too small. These pixel clusters
represent regions in the image (depicted as white rectangles in Fig S2(c)).
Each region corresponds to a predicted nucleus or a set of “closely located”
nuclei, meaning their $(A^{z},A^{\bot})$ values are similar. Each of these
regions is examined to identify local maxima. To account for potential local
variations in pixel values, a minimum distance of 3 pixels ($\sim 3\rm{kHz}$)
between two maxima in the same region is required to consider them as
different nuclei. If two or more nuclei are detected in the same region, we
identify the maxima of each one of them and enclose them in $5\times 5$ red
boxes centered around the maxima (see Fig S2(c)). When a single nucleus is
detected in a region (i. e. there is just one maximum), the entire region is
considered as the detected nucleus, with its centroid serving as the predicted
coupling constants of the nucleus.

Figure S2: (a) True output image taken from the test subset of the low-field
case. 8 nuclei are represented in the image, 2 of them closely located. (b)
Output image predicted by the neural network. (c) Output image after the post-
processing module. The regions corresponding to a single nucleus or, in this
particular case, two closely located nuclei, are represented by white boxes.
In the region where two nuclei are detected (because there are two maxima), we
also see two red boxes, each representing one nucleus. (d) Evaluation of the
prediction, depicting the true nuclei in green boxes. All 8 nuclei are
correctly predicted.

###  II.3 Neural network training

The employed loss function to train the network is Mean Square Error (MSE),
calculated across all the pixels in the output grid, and the optimization is
performed using the Adam optimizer initialized with a learning rate of 0.001.
If there is no improvement in validation loss over 5 epochs, the learning rate
decreases by a factor of 0.7. Training is conducted in batches of 64 samples,
with dataset shuffling after each epoch. The networks undergo training for a
maximum of 250 epochs, employing early stopping with a patience of 20. The
model saved from the epoch with the lowest validation loss is considered the
final model for testing evaluation. The CNN is implemented using the Keras
framework with a TensorFlow backend for training, validation, and testing.

###  II.4 Evaluation

After training, validation, and testing the neural network, image post-
processing is applied to the predicted outputs from the test subset to assess
the model’s performance. Evaluation involves determining the true positives
(TP), false positives (FP), and false negatives (FN) for each sample.
Specifically, “true positive” refers to correctly predicted nuclei, “false
positive” indicates the prediction of non-existent nuclei, and “false
negative” signifies that present nuclei were not predicted. To perform this
evaluation, all nuclei in an image sample are enclosed within bounding boxes:
the nuclei in the true sample (“true nuclei”) are identified in $5\times 5$
green boxes, while the nuclei in the predicted sample (“predicted nuclei”) are
identified through the image post-processing module in white/red boxes, as
described earlier (see Fig. S2(d)). For each white/red box (representing a
predicted nucleus), we calculate the Intersection over Union (IoU) with each
green box (representing the true nuclei), which is defined as the area of
overlap over the area of union; ideally, an IoU of 1 indicates a perfect
match. The green box with the largest IoU is considered the detected true
nucleus. For a nucleus to be considered detected, the IoU between the boxes of
the true and predicted nuclei must exceed 0. In addition to tracking true
positives (TPs) by measuring IoU, we also compute the Mean Absolute Error
(MAE) between the true and predicted coupling constants $A^{z}$ and
$A^{\bot}$. Finally, we assess the input signals reconstructed with all
predicted nuclei by comparing them to the input signals acquired with the true
nuclei employing MAE.

##  III Numerical results

###  III.1 Metrics of the model

Low magnetic field
---
# of true nuclei |  $\rm{P}$ |  $\rm{R}$ |  MAE $A^{z}$ (Hz) |  MAE $A^{\bot}$ (Hz) |  MAE input signal N=32 |  MAE input signal N=256
1 |  0.9341 |  0.9939 |  211.8066 |  261.2769 |  0.0458 |  0.0921
2 |  0.9255 |  0.9848 |  272.2131 |  350.5561 |  0.0595 |  0.1224
3 |  0.8853 |  0.9744 |  349.7452 |  446.5646 |  0.0656 |  0.1290
4 |  0.8289 |  0.9589 |  434.0356 |  557.3587 |  0.0676 |  0.1233
5 |  0.7684 |  0.9435 |  524.2099 |  660.1902 |  0.0666 |  0.1129
6 |  0.7104 |  0.9255 |  613.1820 |  761.0398 |  0.0638 |  0.1010
7 |  0.6562 |  0.9047 |  710.9949 |  863.9525 |  0.0598 |  0.0891
8 |  0.6087 |  0.8789 |  808.9583 |  955.3167 |  0.0560 |  0.0792
9 |  0.5648 |  0.8479 |  909.0699 |  1048.2445 |  0.0523 |  0.0708
10 |  0.5301 |  0.8121 |  1003.8220 |  1135.0094 |  0.0489 |  0.0642
11 |  0.4987 |  0.7702 |  1100.3608 |  1212.2242 |  0.0459 |  0.0586
12 |  0.4738 |  0.7285 |  1202.8350 |  1294.2948 |  0.0433 |  0.0543
13 |  0.4525 |  0.6815 |  1290.1381 |  1365.7002 |  0.0412 |  0.0508
14 |  0.4348 |  0.6342 |  1384.2604 |  1432.3049 |  0.0394 |  0.0479
15 |  0.4194 |  0.5855 |  1470.0856 |  1496.6571 |  0.0378 |  0.0456
16 |  0.4055 |  0.5383 |  1562.0888 |  1558.5923 |  0.0365 |  0.0437
17 |  0.3960 |  0.4925 |  1641.0533 |  1605.6893 |  0.0355 |  0.0423
18 |  0.3879 |  0.4518 |  1718.2080 |  1657.2404 |  0.0348 |  0.0412
19 |  0.3797 |  0.4120 |  1788.7925 |  1692.8111 |  0.0343 |  0.0404
20 |  0.3734 |  0.3749 |  1851.7677 |  1735.7796 |  0.0340 |  0.0399
Table 1: Numerical values from Fig. 3(a)-(c) in the main text: low magnetic
field. These results are calculated separately depending on the number of
nuclei present in the true samples. High magnetic field
---
# of true nuclei |  $\rm{P}$ |  $\rm{R}$ |  MAE $A^{z}$ (Hz) |  MAE $A^{\bot}$ (Hz) |  MAE input signal N=32 |  MAE input signal N=256
1 |  0.8855 |  0.7663 |  159.3502 |  669.6763 |  0.0169 |  0.0202
2 |  0.9524 |  0.7837 |  154.4154 |  710.4753 |  0.0190 |  0.0275
3 |  0.9556 |  0.8417 |  158.6041 |  758.5075 |  0.0212 |  0.0341
4 |  0.9536 |  0.8509 |  166.2072 |  799.9738 |  0.0241 |  0.0406
5 |  0.9462 |  0.8419 |  173.8719 |  853.4260 |  0.0270 |  0.0465
6 |  0.9383 |  0.8317 |  181.4522 |  902.5522 |  0.0297 |  0.0519
7 |  0.9271 |  0.8207 |  193.6377 |  957.8667 |  0.0331 |  0.0571
8 |  0.9166 |  0.8076 |  201.5162 |  1007.4918 |  0.0361 |  0.0620
9 |  0.9041 |  0.7983 |  212.6115 |  1067.0834 |  0.0390 |  0.0663
10 |  0.8933 |  0.7885 |  221.0919 |  1116.6486 |  0.0421 |  0.0709
11 |  0.8817 |  0.7786 |  232.2747 |  1164.1576 |  0.0452 |  0.0746
12 |  0.8696 |  0.7659 |  242.2465 |  1208.1121 |  0.0478 |  0.0779
13 |  0.8582 |  0.7581 |  252.0024 |  1261.3507 |  0.0505 |  0.0811
14 |  0.8472 |  0.7477 |  263.8876 |  1305.2804 |  0.0534 |  0.0844
15 |  0.8364 |  0.7373 |  276.0918 |  1352.9696 |  0.0559 |  0.0871
16 |  0.8283 |  0.7257 |  286.0759 |  1394.1018 |  0.0580 |  0.0895
17 |  0.8168 |  0.7153 |  296.8244 |  1439.9926 |  0.0606 |  0.0917
18 |  0.8098 |  0.7040 |  309.8346 |  1482.3191 |  0.0628 |  0.0938
19 |  0.8000 |  0.6931 |  321.4237 |  1525.2497 |  0.0649 |  0.0958
20 |  0.7921 |  0.6820 |  333.2698 |  1561.6735 |  0.0669 |  0.0975
Table 2: Numerical values from Fig. 3(a)-(c) in the main text: high magnetic
field.

###  III.2 Examples from the main text

Low magnetic field
---
Nucleus |  True $A^{z}$ (Hz) |  Predicted $A^{z}$ (Hz) |  True $A^{\bot}$ (Hz) |  Predicted $A^{\bot}$ (Hz) |  MAE $A^{z}$ (Hz) |  MAE $A^{\bot}$ (Hz)
1 |  -98875.9444 |  -98977.6469 |  22447.4441 |  22149.7736 |  101.7025 |  297.6705
2 |  |  -91381.1482 |  |  77329.0481 |  |
3 |  -82236.4611 |  -81407.0352 |  77605.0795 |  79489.1775 |  829.4259 |  1884.0980
4 |  -62594.5591 |  -62383.3453 |  52589.1205 |  52937.9509 |  211.2138 |  348.8304
5 |  -40809.0586 |  -40134.6354 |  79724.3165 |  78805.7938 |  674.4232 |  918.5228
6 |  |  -31221.3240 |  |  75342.1168 |  |
7 |  -6661.2290 |  |  6511.8053 |  |  |
8 |  618.3637 |  215.3625 |  71233.4150 |  71235.2092 |  403.0012 |  1.7942
9 |  22835.2283 |  18848.5981 |  35535.9281 |  37496.4055 |  3986.6302 |  1960.4774
10 |  16596.4429 |  16086.5253 |  41711.0597 |  45149.3541 |  509.9176 |  3438.2945
11 |  15900.9482 |  15723.3536 |  83833.8436 |  81957.4694 |  177.5945 |  1876.3741
12 |  29856.8923 |  31612.6085 |  64685.9245 |  62537.1901 |  1755.7162 |  2148.7344
13 |  34015.0876 |  35494.7985 |  51611.0833 |  47527.5732 |  1479.7109 |  4083.5101
14 |  41004.5308 |  41864.7914 |  35584.3040 |  36961.9774 |  860.2606 |  1377.6733
15 |  77065.1584 |  |  29807.1735 |  |  |
16 |  |  79676.1586 |  |  19732.8844 |  |
17 |  89884.9243 |  92841.0141 |  97399.2709 |  96192.1903 |  2956.0898 |  1207.0806
18 |  |  94358.1544 |  |  49727.2727 |  |
19 |  97146.7497 |  99163.0711 |  89029.5304 |  86725.7760 |  2016.3214 |  2303.7544
20 |  97293.2625 |  97487.4372 |  37515.6950 |  36848.4848 |  194.1747 |  667.2102
Table 3: Numerical values of the true and predicted coupling constants
$(A^{z},A^{\bot})$, as well as the corresponding MAEs of the example from Fig.
3(d)-(e) in the main text. 14 nuclei are detected (TPs) out of the 16 true
nuclei. 4 nuclei are over-predicted (FPs). High magnetic field
---
Nucleus |  True $A^{z}$ (Hz) |  Predicted $A^{z}$ (Hz) |  True $A^{\bot}$ (Hz) |  Predicted $A^{\bot}$ (Hz) |  MAE $A^{z}$ (Hz) |  MAE $A^{\bot}$ (Hz)
1 |  -89149.1218 |  -89391.4015 |  58528.9955 |  59126.8238 |  242.2796 |  597.8283
2 |  -89250.2135 |  -88996.2634 |  17861.7656 |  17565.9156 |  253.9501 |  295.8500
3 |  -72472.3346 |  -72437.9473 |  76943.9533 |  76196.5106 |  34.3873 |  747.4427
4 |  -71864.3395 |  -71922.1106 |  43449.0304 |  45797.3485 |  57.7711 |  2348.3181
5 |  -67856.0332 |  -67967.1083 |  62388.9558 |  63707.9890 |  111.0751 |  1319.0332
6 |  -43283.2279 |  -43177.4256 |  36742.4227 |  37291.3753 |  105.8023 |  548.9526
7 |  -34559.2966 |  -34673.3668 |  72799.0305 |  72707.0707 |  114.0702 |  91.9598
8 |  -25741.1784 |  -25642.2826 |  41337.3828 |  41100.0937 |  98.8958 |  237.2891
9 |  -21229.8613 |  -20136.9824 |  31394.1066 |  32611.2804 |  1092.8789 |  1217.1738
10 |  -5100.0355 |  -4781.0481 |  63366.8872 |  65145.7431 |  318.9874 |  1778.8560
11 |  |  -3661.1630 |  |  49835.4978 |  |
12 |  38751.9170 |  38627.5640 |  30000.2914 |  28444.7756 |  124.3530 |  1555.5158
13 |  41270.5583 |  41909.5477 |  70786.8143 |  68707.0707 |  638.9895 |  2079.7436
14 |  46661.3024 |  47180.3462 |  47310.4908 |  46332.2110 |  519.0437 |  978.2798
15 |  69741.6539 |  69770.4207 |  29130.9627 |  27272.3312 |  28.7668 |  1858.6315
16 |  76707.7658 |  76469.3030 |  40118.2734 |  40647.3430 |  238.4628 |  529.0696
17 |  87940.3129 |  88358.4590 |  40976.3481 |  40236.5320 |  418.1461 |  739.8161
Table 4: Numerical values of the true and predicted coupling constants
$(A^{z},A^{\bot})$, as well as the corresponding MAEs of the example from Fig.
3(f)-(g) in the main text. All 16 nuclei are detected (TPs). 1 nucleus is
over-predicted (FPs).

## References

  * [1] T. H. Taminiau, J. J. T. Wagenaar, T. Van der Sar, F. Jelezko, V. V. Dobrovitski, and R. Hanson, Detection and control of individual nuclear spins using a weakly coupled electron spin, Phys. Rev. Lett. 109, 137602 (2012).


2022

[1]Arkadiusz Bochniak

[1]Institute of Theoretical Physics, Jagiellonian University, prof. Stanisława
Łojasiewicza 11, Kraków, 30-348, Poland

# Spectral interactions between strings in the Higgs background

<EMAIL_ADDRESS>Andrzej Sitarz<EMAIL_ADDRESS>*

###### Abstract

We derive the exact form of the spectral interaction of two strings mediated
by a constant scalar field using methods derived from noncommutative geometry.
This is achieved by considering a non-product modification of the Connes-Lott
model with two-dimensional manifolds. The analogy with the latter construction
justifies the interpretation of the scalar field as being of Higgs type.
Working in dimension two requires the use of the spectral zeta function
instead of the Wodzicki residue techniques applicable to four-dimensional
models. In the latter case, an analogous non-product geometry construction
leads, for specific choices of metrics, to the so-called ”doubled geometry
models”, which can be thought of as a spectral modification of the Hassan-
Rosen bimetric theory. We find that in dimension two, the interaction term
depends explicitly on zweibeins defining the Dirac operators and only in some
special cases can they be expressed solely using the metrics. The computations
can be performed analytically for an arbitrary choice of zweibeins defining
geometry on the two strings.

###### keywords:

Noncommutative Geometry, Bimetric Theory, Spectral Action

##  1 Introduction

The spectral methods of noncommutative geometry Con95 ; Con96  can be
successfully applied to describe both General Relativity Kas95 ; Con93  and
Standard Model of particle physics together with its potential extensions
ConnesLott ; DunSu12 ; SuBook ; Con90 . The fundamental object in
noncommutative geometry, a spectral triple, is motivated by the observation
made by A. Connes that for any sufficiently regular Riemannian manifold $M$
that allows spinor fields, one can associate a triple consisting of an algebra
$A=C^{\infty}(M)$ of smooth functions on $M$ represented on a Hilbert space
$H=L^{2}(S)$ of square-integrable spinors and the Dirac operator given locally
as $\mathcal{D}=i\gamma^{\mu}(\partial_{\mu}+\omega_{\mu})$, where $\omega$ is
the spin connection on $M$ and $\gamma$’s are the usual generators of the
associated Clifford algebra. By Connes’ reconstruction theorem Con96 ; Con13 ,
any suitably regular triple $(A,H,\mathcal{D})$, with an abelian algebra $A$,
is of the above canonical form. This motivates the notion of a spectral triple
as a basic object that generalizes the notion of geometry to objects that are
discrete spaces, fractals, or that are described by algebras, which are no
longer assumed to be commutative. The data of a spectral triple consists of a
$\ast$-algebra $A$ represented (faithfully) on a Hilbert space $H$ on which
$\mathcal{D}$ is as an essentially self-adjoint operator having compact
resolvent, such that its commutators with elements of the algebra $A$ are
bounded. A set of possible additional structures, like grading and real
structure, can be incorporated into this picture and additional compatibility
conditions between all these elements can be assumed. For a more detailed
discussion see e.g. Con94 ; Li17 .

Out of the above spectral data, one can compute the so-called spectral action
EI18  which is essentially a functional in $\mathcal{D}$. By spectral action
principle, its leading terms in heat kernel expansion, heat  describe the
physical model obtained from the geometry associated with a given spectral
triple. This generalizes Einstein’s idea of equivalence between geometry and
physics (gravity). Indeed, the computation of the (leading terms of) the
spectral action for the canonical spectral triple associated with a given
manifold leads to the Hilbert-Einstein action. This picture was expanded to
include Yang-Mills gauge theories within the framework of almost-commutative
geometry when the spectral triple is a product of the canonical one and
another one with both an algebra and a Hilbert space being finite-dimensional
Con95 ; Con96 . Appropriate choice of the finite part (closely related to the
gauge group one is interested in) allowed for a formulation of the Standard
Model of particle physics within this framework and to study its properties as
well as symmetries lepto99 ; DaAnSi ; AnKuLi .

However, the almost-commutative framework seems not to be the final answer for
all physically relevant questions. In particular, it is very restrictive in
the gravitational sector and it leads (in its bare version) to some unphysical
behavior in particle physics models. Several non-product modifications of
almost-commutative geometries were therefore proposed. Some of them are
formulated at the level of Kasparov modules BoevS ; BoevD , whereas others are
direct modifications motivated by potential applications both for the Standard
Model BoSi20 ; BoSiZa21  as well as for modified gravity theories. In Si19 ,
an approach based on the straightforward modification of the Connes-Lott model
ConnesLott  was proposed. The basic idea was to replace the almost-commutative
geometry $M\times\mathbb{Z}_{2}$ by a direct sum of two copies of $M$ but
considered with different metrics on each summand. This so-called doubled
geometry model was further studied in the case of Friedmann-Lemaître-
Robertson-Walker metrics in BoSi21  and for other classes of metrics also in
Bo22 . Since the resulting action contains interactions between the two
metrics it is natural to ask about potential relations to Hassan-Rosen
bimetric gravity theories HaRo12 ; Akrami13 . We have discussed partially this
aspect in BoSi21  and continued in BoSi22 , where the picture of two four-
dimensional branes interacting effectively by Higgs-like potential was
introduced.

Let us briefly summarize main steps in the construction of the doubled models
with two copies of the spin manifold $M$ of dimension $d>0$. As the Dirac
operator, we take

$\mathcal{D}=\begin{pmatrix}\mathcal{D}_{g}&\gamma\Phi\\\
\gamma\Phi^{\ast}&\mathcal{D}_{h}\end{pmatrix},$ (1)

where $\mathcal{D}_{g}$ (resp. $\mathcal{D}_{h}$) is the standard Dirac
operator corresponding to the Riemannian manifold $(M,g)$ (resp. $(M,h)$),
$\Phi$ is a (constant) field and $\gamma$ is taken so that $\gamma^{2}=\pm 1$
and it anticommutes with the Dirac operators on both sheets. The computation
of all relevant terms of the spectral action is, however, dimension dependent.
The leading term can be expressed using the Wodzicki residue of
$\lvert\mathcal{D}\rvert^{-d}$ and will give the sum of volumes of each copy
of $M$ (with respect to the metric $g$ and $h$, respectively). However, the
next term, which will contain the sum of integrated scalars of curvature for
each copy, will also have an interaction term between the two metric
manifolds. In the case of $d=2$ the explicit computation of these terms
necessitates the use of the zeta function ConTre ; ZaKha .

The tools to compute the corresponding terms of spectral action are based on
the calculus of pseudodifferential operators. We first need to find the
symbols of the operator $\mathcal{D}^{2}$ by replacing each derivative
$\partial_{j}$ with a formal variable $i\xi_{j}$. Let $\mathfrak{a}_{k}$
denotes the part of this symbol that is homogeneous in $\xi_{\mu}^{\prime}s$
with the homogeneity degree $k$. In the next step, we compute the symbols
$\mathfrak{b}_{\bullet}$ for $\mathcal{D}^{-2}$ Gilkey , and it turns out that
only two of them ($\mathfrak{b}_{0}$ and $\mathfrak{b}_{2}$) are enough to
obtain two leading terms of the spectral action.

Motivated by the fact that this approach can produce spectral interactions of
branes, we focus in this paper on the case of two-dimensional branes, the
strings. Our model does not assume any ambient space in which the strings
propagate as the action is fully intrinsic and arises from the scalar field
that links the two strings. Another possible interpretation is that of a
“thick string” where the resulting action is the interaction of its boundaries
(surfaces) in the Higgs background.

We concentrate on the situation where the action comes from the generalized
Dirac operator that involves the Higgs field. Thus, we work with metrics given
in terms of zweibeins. In section 2 we formulate the two-dimensional
Riemannian doubled geometry model in terms of zweibeins and derive the
resulting effective action using spectral methods. Then, in section 3 we
briefly discuss a reformulation of this model in terms of the corresponding
metrics. Finally, in section 4 we shortly discuss the Lorentzian formulation,
interpretation, and present an outlook.

##  2 Formulation in terms of zweibeins

For a two-dimensional Riemannian spinc manifold with a given metric $g$
defined in terms of a zweibein $e^{\mu}_{a}$ (with an inverse metric
$g^{\mu\nu}=e^{\mu}_{a}e^{\nu}_{b}\delta^{ab}$) we consider the Dirac operator
of the form

$\mathcal{D}_{g}=i\sigma^{a}e^{\mu}_{a}\partial_{\mu}-\frac{1}{2}\sigma^{a}\partial_{\mu}(e^{\mu}_{a}),$
(2)

with both $a,b=1,2$ and $\mu=1,2$, and the Pauli matrices $\sigma^{a}$
satisfying $\\{\sigma^{a},\sigma^{b}\\}=2\delta^{ab}I$. Let

$K=\begin{pmatrix}a_{1}&d_{1}\\\ c_{1}&b_{1}\end{pmatrix},$

be the matrix corresponding to the zweibein $e^{\mu}_{a}$, i.e. with entries
$K^{\mu}_{\ a}:=e^{\mu}_{a}$. Since there is no one-to-one correspondence
between metrics and zweibeins (a zweibein determines the metric but a metric
can correspond to a class of zweibeins), in this paper, we treat the zweibein
$e^{\mu}_{a}$, rather than the metric $g$, as a more fundamental object.

In the doubled model we denote the second metric by $h$, the second zweibein
by $f^{\mu}_{a}$ and its corresponding matrix by $L$.

Since we are interested mostly in the form of the interaction potential we can
omit terms with derivatives of zweibeins. Under this assumption, the Dirac
operator for the doubled geometry model takes the form

$\mathcal{D}=i\sigma^{a}A^{\mu}_{a}\partial_{\mu}+\sigma F,$ (3)

with

$A^{\mu}_{a}=\begin{pmatrix}K^{\mu}_{\ a}&0\\\ 0&L^{\mu}_{\
a}\end{pmatrix},\quad F=\begin{pmatrix}0&\Phi\\\ \Phi^{\ast}&0\end{pmatrix},$
(4)

where $\Phi$ is assumed to be a constant, and $\sigma=\chi\sigma^{3}$ with
$\chi$ chosen s.t. $\sigma^{2}=\kappa=\pm 1$. We can now easily compute the
operator $\mathcal{D}^{2}$, and its symbols read

$\begin{split}&\mathfrak{a}_{2}=\begin{pmatrix}(K\xi)^{2}&0\\\
0&(L\xi)^{2}\end{pmatrix}=\begin{pmatrix}\|\xi\|_{g}^{2}&0\\\
0&\|\xi\|_{h}^{2}\end{pmatrix},\\\
&\mathfrak{a}_{1}=[F,A^{\mu}_{a}]\sigma^{a}\sigma\xi_{\mu},\\\
&\mathfrak{a}_{0}=\kappa F^{2},\end{split}$ (5)

where $\xi=\begin{pmatrix}\xi_{1}\\\ \xi_{2}\end{pmatrix}$ is the vector of
symbols of derivatives (i.e. $\partial_{j}\leftrightarrow i\xi_{j}$ for
$j=1,2$).

The computation of the symbols of $\mathcal{D}^{-2}$ simplifies for the
derivative-free case and we get Gilkey :

$\mathfrak{b}_{0}=(\mathfrak{a}_{2}+1)^{-1},\qquad\mathfrak{b}_{2}=-\mathfrak{b}_{0}\mathfrak{a}_{0}\mathfrak{b}_{0}+\mathfrak{b}_{0}\mathfrak{a}_{1}\mathfrak{b}_{0}\mathfrak{a}_{1}\mathfrak{b}_{0}.$
(6)

We remark that the above form of the $\mathfrak{b}_{0}$ symbol is a
consequence of the fact that the manifold is two-dimensional ConTre ; ZaKha .

To proceed with the computation of the spectral action, we first notice that

$\begin{split}&\mathrm{Tr\ }\mathrm{Tr}_{Cl}(\mathfrak{b}_{0}^{2})\\\
&=2\left(\frac{1}{((K\xi)^{2}+1)^{2}}+\frac{1}{((L\xi)^{2}+1)^{2}}\right)\end{split}$
(7)

and

$\int\frac{d^{2}\xi}{((K\xi)^{2}+1)^{2}}=\frac{\pi}{\det(K)},$ (8)

where the trace $\mathrm{Tr}$ is over the discrete degrees of freedom and
$\mathrm{Tr}_{Cl}$ is the trace over the Clifford algebra. Then, the
contribution to the action from this term is simply

$2\pi\left(\frac{1}{\det(K)}+\frac{1}{\det(L)}\right),$

and gives the standard cosmological constant terms for the two metrics.

For the next term in the action, we obtain first,

$\begin{split}&\mathrm{Tr\
}\mathrm{Tr}_{Cl}(-\mathfrak{b}_{0}\mathfrak{a}_{0}\mathfrak{b}_{0})\\\
&=-2\kappa\lvert\Phi\rvert^{2}\left[\frac{1}{\left((K\xi)^{2}+1\right)^{2}}+\frac{1}{\left((L\xi)^{2}+1\right)^{2}}\right]\\\
&=-2\pi\kappa\lvert\Phi\rvert^{2}\left(\frac{1}{\det(K)}+\frac{1}{\det(L)}\right),\end{split}$
(9)

which is just a correction to the previous part.

Finally, the integrand of the remaining part of the potential is,

$\begin{split}&\mathrm{Tr\
}\mathrm{Tr}_{Cl}(\mathfrak{b}_{0}\mathfrak{a}_{1}\mathfrak{b}_{0}\mathfrak{a}_{1}\mathfrak{b}_{0})\\\
&=-2\kappa\mathfrak{b}_{0}[F,A^{\mu}_{a}]\mathfrak{b}_{0}[F,A^{\nu}_{b}]\mathfrak{b}_{0}\delta^{ab}\xi_{\mu}\xi_{\nu},\end{split}$
(10)

which gives

$\begin{split}&\mathrm{Tr\
}\mathrm{Tr}_{Cl}(\mathfrak{b}_{0}\mathfrak{a}_{1}\mathfrak{b}_{0}\mathfrak{a}_{1}\mathfrak{b}_{0})\\\
&=2\kappa\lvert\Phi\rvert^{2}\det(\mathfrak{b}_{0})\mathrm{Tr}(\mathfrak{b}_{0})((K-L)\xi)^{2},\end{split}$
(11)

and, as a result, we obtain the following integration formula

$\begin{split}2\kappa\lvert\Phi\rvert^{2}\int
d^{2}\xi&\left(\frac{((K-L)\xi)^{2}}{((K\xi)^{2}+1)^{2}((L\xi)^{2}+1)}\right.\\\
&\left.+\frac{((K-L)\xi)^{2}}{((K\xi)^{2}+1)((L\xi)^{2}+1)^{2}}\right).\end{split}$
(12)

Let us denote the above contribution (12) to the potential by $V_{1}$. In
order to compute this integral, notice that the second term is just obtained
from the first one by exchanging $K$ with $L$. So, for a moment we concentrate
on the first term only. Let then $S$ be an orthogonal matrix that diagonalizes
$(K^{-1})^{T}L^{T}LK^{-1}$ to $\Lambda^{T}\Lambda$, i.e.

$S^{-1}(K^{-1})^{T}L^{T}LK^{-1}S=\Lambda^{T}\Lambda,$ (13)

and take $P=K^{-1}S$. Here we have used the fact that
$(K^{-1})^{T}L^{T}LK^{-1}$ have positive eigenvalues.

Then changing the coordinates according to $P\eta=\xi$ we obtain

$\begin{split}&\det(K^{-1}S)\int
d^{2}\eta\frac{((S-LK^{-1}S)\eta)^{2}}{(\eta^{2}+1)^{2}((\Lambda\eta)^{2}+1)}\\\
&\equiv\det(K^{-1}S)\int
d^{2}\eta\frac{(W\eta)^{2}}{(\eta^{2}+1)^{2}((\Lambda\eta)^{2}+1)},\end{split}$
(14)

where we have introduced $W=S-LK^{-1}S$.

As the denominator is symmetric with respect to $\eta$, only the parts with
$\eta_{1}^{2},\eta_{2}^{2}$ in the numerator will enter, so the considered
part of the potential reads

$\begin{split}&V_{1}=2\kappa\lvert\Phi\rvert^{2}\det(K^{-1}S)\\\ &\times\int
d^{2}\eta\left((W_{11}^{2}+W_{21}^{2})\eta_{1}^{2}+(W_{22}^{2}+W_{12}^{2})\eta_{2}^{2}\right)\\\
\times&\left(\frac{1}{(\eta^{2}+1)^{2}\left((\Lambda_{1}\eta_{1})^{2}+(\Lambda_{2}\eta_{2})^{2}+1\right)}\right.\\\
&\left.+\frac{1}{(\eta^{2}+1)\left((\Lambda_{1}\eta_{1})^{2}+(\Lambda_{2}\eta_{2})^{2}+1\right)^{2}}\right),\end{split}$
(15)

where we used the fact that $(W\eta)^{2}=\eta^{T}W^{T}W\eta$ is symmetric with
respect to the interchange $K\leftrightarrow L$. This can be written as

$\begin{split}2\kappa\lvert\Phi\rvert^{2}\det(K^{-1}S)&\left[(W_{11}^{2}+W_{21}^{2})I_{1}(1,1,\Lambda_{1},\Lambda_{2})\right.\\\
+&\left.(W_{22}^{2}+W_{12}^{2})I_{1}(1,1,\Lambda_{2},\Lambda_{1})\right.\\\
+&\left.(W_{11}^{2}+W_{21}^{2})I_{1}(\Lambda_{1},\Lambda_{2},1,1)\right.\\\
+&\left.(W_{22}^{2}+W_{12}^{2})I_{1}(\Lambda_{2},\Lambda_{1},1,1)\right],\end{split}$
(16)

where

$\begin{split}&I_{1}(a,b,c,d)\\\
&=\int_{\mathbb{R}^{2}}\frac{\xi_{1}^{2}d\xi_{1}d\xi_{2}}{(a^{2}\xi_{1}^{2}+b^{2}\xi_{2}^{2}+1)^{2}(c^{2}\xi_{1}^{2}+d^{2}\xi_{2}^{2}+1)}\\\
&=\frac{c}{a}\frac{\pi}{(c^{2}-a^{2})(bc+ad)}+\frac{\pi}{(c^{2}-a^{2})^{3/2}(b^{2}-d^{2})^{1/2}}\\\
&\hskip
71.13188pt\times\left[\arcsin\left(\frac{a\sqrt{b^{2}-d^{2}}}{\sqrt{b^{2}c^{2}-a^{2}d^{2}}}\right)\right.\\\
&\hskip
71.13188pt\left.-\arcsin\left(\frac{c\sqrt{b^{2}-d^{2}}}{\sqrt{b^{2}c^{2}-a^{2}d^{2}}}\right)\right].\end{split}$
(17)

Since

$\begin{split}I_{1}(1,1,\Lambda_{1},\Lambda_{2})+I_{1}&(\Lambda_{1},\Lambda_{2},1,1)\\\
&=\frac{\pi}{\Lambda_{1}+\Lambda_{2}}\,\frac{1}{\Lambda_{1}},\end{split}$ (18)

we finally get

$\begin{split}V_{1}&=\frac{2\pi\kappa\lvert\Phi\rvert^{2}\det(K^{-1}S)}{\Lambda_{1}+\Lambda_{2}}\\\
&\times\left(\frac{W_{11}^{2}+W_{21}^{2}}{\Lambda_{1}}+\frac{W_{22}^{2}+W_{12}^{2}}{\Lambda_{2}}\right)\\\
&=\frac{2\pi\kappa\lvert\Phi\rvert^{2}\det(K^{-1}S)}{\mathrm{Tr}(\Lambda)}\mathrm{Tr}(W^{T}W\Lambda^{-1}).\end{split}$
(19)

But since $S$ is an orthogonal matrix, this expression can be further reduced
to

$2\pi\kappa\lvert\Phi\rvert^{2}\frac{\mathrm{Tr}(W^{T}W\Lambda^{-1})}{\mathrm{Tr}(\Lambda)\det(K)}.$
(20)

Let us now express (20) in terms of invariants of the matrix $X=LK^{-1}$.
First, by (13) we have

$\mathrm{Tr}(\Lambda)=\mathrm{Tr}((X^{T}X)^{1/2}).$ (21)

Next, again by (13) and the fact that $X^{T}X$ is positive (hence
$\Lambda^{T}\Lambda$ has positive eigenvalues), we have

$\begin{split}\Lambda^{-1}&=(S^{-1}X^{T}XS)^{-1/2}=S^{-1}(X^{T}X)^{-1/2}S\\\
&=S^{T}(X^{T}X)^{-1/2}S,\end{split}$ (22)

and therefore

$\begin{split}W^{T}W\Lambda^{-1}&=S^{T}(X^{T}X)^{-1/2}S\\\
&-S^{T}X(X^{T}X)^{-1/2}S\\\ &-S^{T}X^{T}(X^{T}X)^{-1/2}S\\\
&+S^{T}(X^{T}X)^{1/2}S,\end{split}$ (23)

so that

$\begin{split}\mathrm{Tr}(W^{T}W\Lambda^{-1})&=\mathrm{Tr}((X^{T}X)^{-1/2})\\\
&-\mathrm{Tr}(X(X^{T}X)^{-1/2})\\\ &-\mathrm{Tr}(X^{T}(X^{T}X)^{-1/2})\\\
&+\mathrm{Tr}((X^{T}X)^{1/2}).\end{split}$ (24)

Moreover, since $X^{T}X$ has only positive eigenvalues, we also have

$\begin{split}(X(X^{T}X)^{-1/2})^{T}&=((X^{T}X)^{-1/2})^{T}X^{T}\\\
&=\left[\left((X^{T}X)^{1/2}\right)^{T}\right]^{-1}X^{T}\\\
&=\left[\left((X^{T}X)^{T}\right)^{1/2}\right]^{-1}X^{T}\\\
&=(X^{T}X)^{-1/2}X^{T}\end{split}$ (25)

and therefore,

$\begin{split}\mathrm{Tr}\left(X^{T}(X^{T}X)^{-1/2}\right)&=\mathrm{Tr}\left((X(X^{T}X)^{-1/2})^{T}\right)\\\
&=\mathrm{Tr}(X(X^{T}X)^{-1/2}),\end{split}$ (26)

so that

$\begin{split}\mathrm{Tr}(W^{T}W\Lambda^{-1})&=\mathrm{Tr}((X^{T}X)^{-1/2})\\\
&-2\,\mathrm{Tr}(X(X^{T}X)^{-1/2})\\\
&+\mathrm{Tr}((X^{T}X)^{1/2}).\end{split}$ (27)

As a result,

$\begin{split}&V_{1}=\frac{2\pi\kappa\lvert\Phi\rvert^{2}}{\det(K)}\cdot\frac{1}{\mathrm{Tr}((X^{T}X)^{1/2})}\\\
&\times\left[\mathrm{Tr}((X^{T}X)^{-1/2})-2\,\mathrm{Tr}(X(X^{T}X)^{-1/2})\right.\\\
&\left.\hskip 14.22636pt+\mathrm{Tr}((X^{T}X)^{1/2})\right].\end{split}$ (28)

Since for the $2\times 2$ matrix $(X^{T}X)^{1/2}$ we have

$\begin{split}\mathrm{Tr}((X^{T}X)^{-1/2})&=\frac{\mathrm{Tr}((X^{T}X)^{1/2})}{\det((X^{T}X)^{1/2})}\\\
&=\frac{\mathrm{Tr}((X^{T}X)^{1/2})}{\lvert\det(X)\rvert},\end{split}$ (29)

this expression can be further rewritten as

$\begin{split}&V_{1}=\frac{2\pi\kappa|\Phi\rvert^{2}}{\det(K)}\\\
&\times\left(1+\frac{1}{\lvert\det(X)\rvert}-2\frac{\mathrm{Tr}(X(X^{T}X)^{-1/2})}{\mathrm{Tr}((X^{T}X)^{1/2})}\right).\end{split}$
(30)

In order to proceed further, we use the fact Levinger  that for a $2\times 2$
positive matrix $Y$ we have

$\sqrt{Y}=\frac{Y+\sqrt{\det(Y)}I}{\sqrt{\mathrm{Tr}(Y)+2\sqrt{\det(Y)}}},$
(31)

and hence

$\mathrm{Tr}(\sqrt{Y})=\sqrt{\mathrm{Tr}(Y)+2\sqrt{\det(Y)}}.$ (32)

Furthermore,

$\begin{split}&Y^{-1/2}=\sqrt{Y^{-1}}\\\
&=\frac{Y^{-1}+\sqrt{\det(Y^{-1})}I}{\sqrt{\mathrm{Tr}(Y^{-1})+2\sqrt{\det(Y^{-1})}}}\\\
&=\frac{Y^{-1}+\frac{1}{\sqrt{\det(Y)}}I}{\sqrt{\frac{\mathrm{Tr}(Y)}{\det(Y)}+\frac{2}{\sqrt{\det(Y)}}}}\\\
&=\frac{\sqrt{\det(Y)}Y^{-1}+I}{\sqrt{\mathrm{Tr}(Y)+2\sqrt{\det(Y)}}}.\end{split}$
(33)

Therefore,

$\mathrm{Tr}(XY^{-1/2})=\frac{\sqrt{\det(Y)}\mathrm{Tr}(XY^{-1})+\mathrm{Tr}(X)}{\sqrt{\mathrm{Tr}(Y)+2\sqrt{\det(Y)}}}$
(34)

and

$\frac{\mathrm{Tr}(XY^{-1/2})}{\mathrm{Tr}(Y^{1/2})}=\frac{\sqrt{\det(Y)}\mathrm{Tr}(XY^{-1})+\mathrm{Tr}(X)}{\mathrm{Tr}(Y)+2\sqrt{\det(Y)}}.$
(35)

But in our case $Y=X^{T}X$, so that
$XY^{-1}=X(X^{T}X)^{-1}=XX^{-1}(X^{T})^{-1}=(X^{T})^{-1}$ and hence
$\mathrm{Tr}(XY^{-1})=\mathrm{Tr}(X^{-1})=\frac{\mathrm{Tr}(X)}{\det(X)}$.
Furthermore, $\sqrt{\det(Y)}=\lvert\det(X)\rvert$ and therefore

$\frac{\mathrm{Tr}(XY^{-1/2})}{\mathrm{Tr}(Y^{1/2})}=\frac{(1+\mathrm{sgn}(\det(X)))\,\mathrm{Tr}(X)}{\mathrm{Tr}(X^{T}X)+2\lvert\det(X)\rvert}.$
(36)

For $\det(X)<0$ this term vanishes identically. On the other hand, for
$\det(X)>0$ we get

$\begin{split}V_{1}&=\frac{2\pi\kappa\lvert\Phi\rvert^{2}}{\det(K)}\\\
\times&\left(1+\frac{1}{\det(X)}-\frac{4\,\mathrm{Tr}(X)}{\mathrm{Tr}(X^{T}X)+2\det(X)}\right).\end{split}$
(37)

Observe that the first two terms are, in fact, again of the same type as (9),
however, with an opposite sign. Therefore, in the end, the additional term
proportional to the volumes of the manifolds that is proportional to
$\lvert\Phi\rvert^{2}$ vanishes due to this cancellation. What remains as an
effective interaction between the zweibeins on the two worldsheets is

$V_{\mathrm{int}}=-\frac{8\pi\kappa\lvert\Phi\rvert^{2}}{\det(K)}\left(\frac{\mathrm{Tr}(X)}{\mathrm{Tr}(X^{T}X)+2\det(X)}\right).$
(38)

This can be further simplified using the fact that

$\det(X)=\frac{1}{2}\left(\mathrm{Tr}(X)^{2}-\mathrm{Tr}(X^{2})\right).$

Note that although the expression above uses $K$ and $X$ it can be
equivalently rewritten using $L$ and $X^{-1}$ as it is completely symmetric.

We remark that the above cancellation was possible due to the fact that we are
working in two dimensions. Indeed, the first term in (37) has its form because
for a $2\times 2$ matrix the trace of its inverse can be expressed in terms of
the trace of the original matrix and its determinant, as it was done in
Eq.(29). As a result, in two dimensions, the contribution to the action from
$\mathfrak{b}_{2}$ does not affect the effective cosmological constant, in
contrast to dimension four BoSi21 . Moreover, the interaction is non-zero only
if $\det(X)$ is positive, i.e., both $K$ and $L$ have determinants of the same
signs.

Before we discuss the dependence of the above term on the metrics let us
consider a special case, of diagonal zweibeins.

###  2.1 A special case: diagonal constant zweibeins

We consider now the diagonal case, i.e. with $K=\mathrm{diag}(a_{1},b_{1})$
and $L=\mathrm{diag}(a_{2},b_{2})$. In such a situation we have

$X=\mathrm{diag}\left(\frac{a_{2}}{a_{1}},\frac{b_{2}}{b_{1}}\right),$

so that

$\begin{split}\mathrm{Tr}(X)=\frac{a_{1}b_{2}+a_{2}b_{1}}{a_{1}b_{1}},\quad&\mathrm{Tr}(X^{T}X)=\frac{a_{1}^{2}b_{2}^{2}+a_{2}^{2}b_{1}^{2}}{a_{1}^{2}b_{1}^{2}},\\\
\det(X)=&\frac{a_{2}b_{2}}{a_{1}b_{1}},\end{split}$ (39)

and therefore if we assume that $\det(X)>0$ then

$V_{\mathrm{int}}=-\frac{8\pi\kappa\lvert\Phi\rvert^{2}}{a_{1}b_{2}+a_{2}b_{1}}.$
(40)

##  3 Formulation in terms of metrics

The derivation of the interaction terms between the two worldsheets was
expressed in the previous section directly in terms of zweibeins, which define
the Dirac operators. However, it is interesting to see which part of it can be
rewritten explicitly in terms of the metrics. In this section we discuss this
possibility, using the metrics $g,h$ on the two layers of the doubled model.

First of all, observe that if a zweibein is given by the matrix

$K=\begin{pmatrix}a_{1}&d_{1}\\\ c_{1}&b_{1}\end{pmatrix},$ (41)

then we have

$g^{-1}=\begin{pmatrix}a_{1}^{2}+d_{1}^{2}&a_{1}c_{1}+b_{1}d_{1}\\\
a_{1}c_{1}+b_{1}d_{1}&b_{1}^{2}+c_{1}^{2}\end{pmatrix},$ (42)

and the metric

$\begin{split}g=&\frac{1}{(a_{1}b_{1}-c_{1}d_{1})^{2}}\\\
&\times\begin{pmatrix}b_{1}^{2}+c_{1}^{2}&-(a_{1}c_{1}+b_{1}d_{1})\\\
-(a_{1}c_{1}+b_{1}d_{1})&a_{1}^{2}+d_{1}^{2}\end{pmatrix}.\end{split}$ (43)

Therefore,

$\det(K)=a_{1}b_{1}-c_{1}d_{1}=\sqrt{\det(g^{-1})},$ (44)

and similarly for the matrix $L$ and the metric $h$. Next,

$X=\sqrt{\det(g)}\begin{pmatrix}a_{2}b_{1}-c_{1}d_{2}&a_{1}d_{2}-a_{2}d_{1}\\\
b_{1}c_{2}-c_{1}b_{2}&a_{1}b_{2}-d_{1}c_{2}\end{pmatrix}$ (45)

and

$\begin{split}\det(X)&=\frac{\det(L)}{\det(K)}=\frac{\sqrt{\det(h^{-1})}}{\sqrt{\det(g^{-1})}}\\\
&=\sqrt{\det(gh^{-1})}=\det\left(\sqrt{gh^{-1}}\right).\end{split}$ (46)

Therefore, the only obstruction to rephrasing the action in terms of metrics
rather than zweibeins is given by the term that contains a trace of the matrix
$X$ as well as the trace of $X^{T}X$.

If the zweibeins are such that these quantities are expressible in terms of
invariants of $g$, $h$, and/or $gh^{-1}$, then the full action can be
unambiguously written on the level of metrics. However, in the most general
case, we are dealing with interactions between zweibeins rather than metrics.
This is in contrast to the standard bimetric theory, which was formulated from
the beginning using metrics. The vielbein version was shown HiRo  to be
dynamically equivalent to the usual bimetric theory, however, that is not
necessarily true for multi-metric models (see HaMay ). Of course, this is a
consequence of the assumed model of interactions between the two vielbeins and
the basic structure where spinor fields and vielbeins are fundamental objects.
It is interesting whether there exists a model of the full Dirac operator,
where the effective potential depends only on the metrics.

###  3.1 A special case: diagonal constant zweibeins

In the case of diagonal zweibeins, from the form of the principal symbol, we
easily read that the metrics are such that their inverses are of the form
$\mathrm{diag}(a^{2},b^{2})$, i.e., they are also diagonal. In this case
$\sqrt{\det(g)}=\frac{1}{a_{1}b_{1}}$ and
$\sqrt{\det(h)}=\frac{1}{a_{2}b_{2}}$.

Let us consider the matrix $\mathbb{X}=\sqrt{g^{-1}h}$. Its eigenvalues are
$x=\frac{a_{1}}{a_{2}}$ and $y=\frac{b_{1}}{b_{2}}$, and therefore,

$\begin{split}V_{\mathrm{int}}&=-8\pi\kappa\lvert\Phi\rvert^{2}\sqrt{\det(g)}\underbrace{\frac{xy}{x+y}}_{V\left(\sqrt{g^{-1}h}\right)}\\\
&=-8\pi\kappa\lvert\Phi\rvert^{2}\sqrt{\det(h)}V\left(\sqrt{h^{-1}g}\right).\end{split}$
(47)

##  4 Lorentzian formulation and conclusions

Since the derivation and applicability of the spectral methods rely on a
purely Riemannian framework, the final action describes a Riemannian model.
However, we can use the standard Wick rotation argument to make contact with
the Lorentzian formulation. The procedure is as follows. Since we started with
the modified Connes-Lott model, the effective interaction originating from the
field $\Phi$ has the character of a Higgs-like one. The two-dimensional
Riemannian manifold we started with is now replaced by a $(1+1)$-dimensional
string, and we end up with a model of two Lorentzian strings interacting via a
Higgs-like field. The interaction is given at the level of corresponding
zweibeins for these two strings. This is the two-dimensional generalization of
the picture with four-dimensional branes we discussed previously in BoSi22  to
make a comparison between doubled geometry models Si19 ; BoSi21  and Hassan-
Rosen bimetric gravity theories HaRo12 .

Contrary to the four-dimensional situation, in dimension two we compute the
zeta function rather than a residue, and the resulting potential can be
computed analytically for generic zweibeins. The resulting interaction is non-
zero only if the matrices corresponding to the zweibeins have determinants of
the same signs, and is then expressed in terms of the invariants of the matrix
$X=LK^{-1}$ with parts of the result expressible only in terms of metrics.
This is analogous to the formulation of bimetric gravity in terms of vielbeins
and metrics HiRo  and similar to certain choices for vielbeins that ensure the
existence of real square root $\sqrt{g^{-1}h}$ DeMoZa . We postpone the
detailed discussion of this aspect for future research.

Acknowledgments A. B. thanks S.F. Hassan for insightful discussions and for
stating the question that motivated this study. The authors acknowledge the
support of the National Science Centre, Poland, Grant No. 2020/37/B/ST1/01540.

Competing interests The authors declare no competing interests.

Author contributions Both authors contribute equally to the work as well as to
the writing of the manuscript.

Data availability All data generated or analyzed during this study are
included in the manuscript.

## References

  * (1) A. Connes, Noncommutative geometry and reality, J. Math. Phys. 36, 6194 (1995).
  * (2) A. Connes, Gravity coupled with matter and foundation of noncommutative geometry, Commun. Math. Phys. 182, 155 (1996).
  * (3) D. Kastler, The dirac operator and gravitation, Commun. Math. Phys. 166, 633–643 (1995).
  * (4) A. Connes, Noncommutative geometry in physics, preprint IHES/M/93/32 (1993).
  * (5) A. Connes, and J. Lott, Particle models and noncommutative geometry, Nucl. Phys. Proc. Suppl. 18B, 29–47 (1991).
  * (6) K. van den Dungen, and W.D. van Suijlekom, Particle physics from almost-commutative spacetimes, Rev. Math. Phys. 24 (9), 1230004 (2012).
  * (7) W.D. van Suijlekom, Noncommutative Geometry and Particle Physics (Springer, Dordrecht 2015).
  * (8) A. Connes, Essay on physics and noncommutative geometry, in: The interface of mathematics and particle physics (Oxford, 1988), vol. 24 of Inst. Math. Appl. Conf. Ser. New Ser. 9–48, Oxford Univ. Press, New York (1990).
  * (9) A. Connes, On the spectral characterization of manifolds, J. Noncommut. Geom. 7, 1-82 (2013).
  * (10) A. Connes, Noncommutative Geometry (Academic Press, Boca Raton, 1994).
  * (11) F. Lizzi, Noncommutative Geometry and Particle Physics, Proceedings of Corfu Summer Institute 2017 ”Schools and Workshops on Elementary Particle Physics and Gravity” — PoS(CORFU2017) (2018).
  * (12) M. Eckstein, and B. Iochum, Spectral action in noncommutative geometry, in Springer Briefs in Mathematical Physics 27, (Springer, Cham, 2018).
  * (13) D.V. Vassilevich, Heat kernel expansion: user’s manual, Phys. Rept. 388, 279 (2003).
  * (14) M. Paschke, F. Scheck, and A. Sitarz, Can (noncommutative) geometry accommodate leptoquarks?, Phys. Rev. D 59, 035003 (1999).
  * (15) L. Dabrowski, F. D’Andrea, and A. Sitarz, The Standard Model in noncommutative geometry: fundamental fermions as internal forms, Lett. Math. Phys. 108, 1323 (2018) [Erratum ibid. 109, 2585 (2019)].n
  * (16) F. D’Andrea, M.A. Kurkov, and F. Lizzi, Wick rotation and fermion doubling in noncommutative geometry, Phys. Rev. D 94, 025030 (2016).
  * (17) J. Boeijink, and W.D. van Suijlekom, The noncommutative geometry of Yang–Mills fields, J. Geom. Phys. 61, 1122–1134 (2011).
  * (18) J. Boeijink, and K. van den Dungen, On globally non-trivial almost-commutative manifolds, J. Math. Phys. 55, 103508 (2014).
  * (19) A. Bochniak, and A. Sitarz, Spectral geometry for the standard model without fermion doubling, Phys. Rev. D 101 075038 (2020).
  * (20) A. Bochniak, A. Sitarz, and P. Zalecki, Spectral action and the electroweak $\theta$-terms for the Standard Model without fermion doubling, J. High Energ. Phys. 2021, 142 (2021).
  * (21) A. Sitarz, On almost commutative Friedmann-Lemaître-Robertson-Walker geometries, Class. Quant. Grav. 36, 195007 (2019).
  * (22) A. Bochniak, and A. Sitarz, Stability of Friedmann-Lemaître-Robertson-Walker solutions in doubled geometries, Phys. Rev. D 103 044041 (2021).
  * (23) A. Bochniak, Towards modified bimetric theories within non-product spectral geometry, J. Phys. A: Math. Theor. 55 414006 (2022).
  * (24) S.F. Hassan, and R. A. Rosen, Bimetric gravity from ghost-free massive gravity, J. High Energ. Phys. 2012, 126 (2012).
  * (25) Y. Akrami, T.S. Koivisto, D.F. Mota, and M. Sandstad, Bimetric gravity doubly coupled to matter: theory and cosmological implications, J. Cosmol. Astropart. Phys. 10, 046 (2013).
  * (26) A. Bochniak, and A. Sitarz, Spectral interaction between universes, J. Cosmol. Astropart. Phys. 04, 055 (2022).
  * (27) A. Connes, and P. Tretkoff, The Gauss-Bonnet Theorem for the noncommutative two torus, arXiv:0910.0188.
  * (28) F.F. Zadeh, and M. Khalkhali, The Gauss-Bonnet theorem for noncommutative two tori with a general conformal structure, J. Noncommut. Geom. 6, 457–-480 (2012).
  * (29) P. Gilkey, Invariance theory, the heat equation, and the Atiyah-Singer index theorem, Mathematics Lecture Series 11, Publish or Perish (1984).
  * (30) B. W. Levinger, The Square Root of a $2\times 2$ Matrix, Math. Mag. 53, 222–-224 (1980).
  * (31) K. Hinterbichler, and R.A. Rosen, Interacting spin-2 fields, J. High Energ. Phys. 2012, 47 (2012).
  * (32) S. F. Hassan and A. Schmidt-May, Interactions of multiple spin-$2$ fields beyond pairwise couplings, Phys. Rev. Lett. 122, 251101 (2019).
  * (33) C. Deffayet, J. Mourad, and G. Zahariade, A note on “symmetric” vielbeins in bimetric, massive perturbative and non perturbative gravities, J. High Energ. Phys. 2013, 86 (2013).


# What and How of Machine Learning Transparency: Building Bespoke
Explainability Tools with Interoperable Algorithmic Components

Kacper Sokol  Alexander Hepburn  Raul Santos-Rodriguez  Peter Flach

††margin:  Preprint Resources • https://github.com/openjournals/jose-
reviews/issues/175https://doi.orgReview • https://github.com/fat-
forensics/Surrogates-Tutorialhttps://doi.orgRepository License
Authors of papers retain copyright and release the work under a Creative
Commons Attribution 4.0 International License
(http://creativecommons.org/licenses/by/4.0/https://doi.orgCC BY 4.0).

## Summary

Explainability techniques for data-driven predictive models based on
artificial intelligence and machine learning algorithms allow us to better
understand the operation of such systems and help to hold them accountable
(Sokol & Flach, 2021a). New transparency approaches are developed at breakneck
speed, enabling us to peek inside these black boxes and interpret their
decisions. Many of these techniques are introduced as monolithic tools, giving
the impression of one-size-fits-all and end-to-end algorithms with limited
customisability. Nevertheless, such approaches are often composed of multiple
interchangeable modules that need to be tuned to the problem at hand to
produce meaningful explanations (Sokol et al., 2019). This paper introduces a
collection of hands-on training materials – slides, video recordings and
Jupyter Notebooks – that provide guidance through the process of building and
evaluating bespoke modular surrogate explainers for tabular data. These
resources cover the three core building blocks of this technique:
interpretable representation composition, data sampling and explanation
generation (Sokol et al., 2019).

## Modular Surrogate Explainers

The training materials introduce the concept of _modular_ explainability
algorithms using the example of surrogate explainers for tabular data. This
separation of functionally independent building blocks allows us to consider
the influence of each component, and their interdependence, on the robustness
and faithfulness of the final explainer. To this end, we review a collection
of techniques to evaluate the quality of the modules and their overall
effectiveness. These metrics can guide the parameterisation of the entire
explainability algorithm, providing an opportunity to tune it to the problem
at hand. All of these insights demonstrate that while surrogate explainers are
model-agnostic and post-hoc – i.e., they work with any black box and can be
retrofitted into pre-existing predictive models, thus making them a popular
choice for explaining black-box predictions (Ribeiro et al., 2016) – using
off-the-shelf explainability approaches may result in subpar performance for
individual use cases (Rudin, 2019). Therefore, understanding how to build a
bespoke surrogate explainer that is suitable for a particular situation is a
prerequisite for trustworthy and meaningful explainability of data-driven
systems and their decisions.

Prior to diving into the practicalities of composing surrogate explainers, the
training materials introduce the concept of algorithmic explainability of
predictive models and discuss the fundamental ideas behind surrogates for
text, image and tabular data. This theoretical overview is followed by a brief
presentation of the software used for the hands-on modules; FAT
Forensics111https://fat-forensics.org/ is an open source Python package
designed for inspecting selected fairness, accountability and _transparency_
aspects of data (and their features), _models_ and _predictions_ (Sokol et
al., 2020, 2022). Having covered the basics, the practical coding resources
focus on the three building blocks of surrogate explainers for tabular data
identified by the bLIMEy – build LIME yourself (Sokol et al., 2019) – meta-
algorithm:

  * •

interpretable (data) representation composition;

  * •

data sampling; and

  * •

explanation generation (interpretable feature selection, data sample
weighting, surrogate model training and explanation extraction).

These learning modules review some of the interoperable algorithmic components
available at each step, discuss their pros and cons for a range of
applications, guide through their optimal selection strategies and propose
suitable evaluation criteria – see Figure 1. In particular, interpretable
representations are built with quartile discretisation and decision trees
(Sokol & Flach, 2020b); data are generated with Gaussian and mixup sampling
(Sokol et al., 2019; Zhang et al., 2018); and explanations are extracted from
linear and tree-based surrogate models (Sokol & Flach, 2020a). Notably, these
choices determine the type, role and quality of the resulting explanations
composed for black-box predictions. Therefore, these hands-on materials
illustrate how such interoperable algorithmic building blocks behave in
various scenarios and demonstrate how to use these components to configure
robust explainers with well-known properties and failure modes based on first-
hand observations and a collection of quantitative evaluation metrics and
validation techniques.

Figure 1: Overview of surrogate explainers modularity listing components
specific to tabular data.

The introduction to algorithmic explainability; the theoretical overview of
surrogate explainers for text, image and tabular data; and the outline of FAT
Forensics are presented in a collection of _slides_ and _instructional video
recordings_. The hands-on materials are delivered with _Jupyter Notebooks_
that interweave textual guidance, code examples and informative plots. All the
insights learnt throughout the practical exercises enable the tutees to create
robust surrogate explainers for an arbitrary black-box predictive model built
for their own tabular data set. The training resources are designed to appeal
and be accessible to an audience with a wide range of backgrounds and
experiences. Active participation in the practical part requires basic
familiarity with Python programming and access to a computer connected to the
Internet, which enables execution of the Jupyter Notebooks online without
installing any software on a personal machine.

These training materials were used to deliver a hands-on tutorial – of the
same title – at the 2020 European Conference on Machine Learning and
Principles and Practice of Knowledge Discovery in Databases (ECML-
PKDD)222https://events.fat-forensics.org/2020_ecml-pkdd/, the recordings of
which are available on
YouTube333https://www.youtube.com/playlist?list=PLgdhPOmeUNm0H2XTQECK3wabnDohZURLK.
Moreover, they inspired a number of interactive sessions at various summer
schools aimed at doctoral students in artificial intelligence and machine
learning, as well as undergraduate lectures, academic presentations and
invited talks. The slides, extra hands-on resources and video recordings of
some of these events are available on the FAT Forensics Events
website444https://events.fat-forensics.org/. The new teaching
materials555https://github.com/fat-forensics/resources/ additionally cover
surrogates for image data – focusing on the influence of segmentation
granularity and occlusion colour on the trustworthiness of the resulting
explanations (Sokol & Flach, 2020b) – and touch upon other explainers such as
_permutation importance_ (Breiman, 2001), _individual conditional expectation_
(Goldstein et al., 2015) and _partial dependence_ (Friedman, 2001).

Notably, these sessions propelled the improvement, evolution and expansion of
the training resources. In particular, the programming-focused exercises have
been complemented by _no-code_ Jupyter Notebooks that enable interactive
experimentation with the explainers through intuitive _Jupyter Widgets_ , thus
allowing to better engage with the audience in a limited time. The same
strategy has been employed for the slides – by embedding interactive examples
based on widgets – to which end they have been built with
RISE666https://rise.readthedocs.io/. From our experience, the teaching became
much more effective when the ubiquitous PDF slides and Jupyter Notebook
programming exercises were replaced with and/or enriched by formats supporting
seamless interaction with the taught material (in our case achieved through
widgets). This exploration of alternative technologies for building training
resources has also inspired a prototype of a new publishing workflow, where
multiple artefacts such as online documents, slides and computational
notebooks can be composed from a unified collection of source materials (Sokol
& Flach, 2021b).

## Statement of Need

The training resources described by this paper introduce a novel learning
paradigm for algorithmic explainability of data-driven predictive systems
based on artificial intelligence and machine learning techniques. Instead of
treating these tools as end-to-end, monolithic entities whose configuration is
only facilitated through the parameters exposed by their developers, these
educational materials look into their modularity to identify atomic and
functionally interoperable building blocks. By decomposing explainers into
their core elements we can better understand their role and configure them for
the application at hand. Within this purview, such techniques are diagnostic
tools that _only_ become explainers when their properties and interpretation
of their outputs are well understood and designed accordingly. Therefore, to
engender trust in data-driven predictive systems, the employed explainability
approaches must be trustworthy themselves in the first place – the learning
objective underlying the interactive coding exercises. The training materials
achieve these goals by supporting the following learning outcomes specifically
for surrogate explainers (which were chosen because of their flexibility and
popularity):

  * •

identify self-contained algorithmic components of the explainer and understand
their functions;

  * •

connect these building blocks to the explainability requirements unique to the
investigated predictive system;

  * •

select appropriate algorithmic components and tune them to the problem at
hand;

  * •

evaluate these building blocks (in this specific context) independently and
when joined together to form the final explainer; and

  * •

interpret the resulting explanations in view of the uncovered properties and
limitations of the bespoke explainability algorithm.

The modularity and diversity of these training materials – slides, video
recordings and Jupyter Notebooks – allow them to be adapted or directly
incorporated into a course on explainable artificial intelligence and
interpretable machine learning, or form the basis of a range of educational
resources such as practical training sessions and conference tutorials. The
module can be taught as is – reusing the slides and computational notebooks –
with either bespoke tuition or by following the prerecorded videos.
Alternatively, the narration, figures and results presented within the
notebooks may be shaped into tailor-made teaching materials. The hands-on
resources can also become a standalone case study supplementing relevant
explainability and interpretability courses. The comprehensive and in-detail
presentation of the topic, covering both the underlying theory and practical
aspects, is suitable for and accessible to undergraduate and postgraduate
students, researchers as well as engineers and data scientists interested in
the subject. This module fills a gap in educational materials dealing with
artificial intelligence and machine learning transparency by focusing on
understanding of the inner workings of these techniques and the influence of
their building blocks on the robustness, veracity and comprehensibility of
explanatory insights into black-box predictive models.

## Acknowledgements

This work was partially supported by TAILOR (Trustworthy AI through
Integrating Learning, Optimisation and Reasoning), a project funded by EU
Horizon 2020 research and innovation programme under GA No 952215; the UKRI
Turing AI Fellowship EP/V024817/1; and the ARC Centre of Excellence for
Automated Decision-Making and Society, funded by the Australian Government
through the Australian Research Council (project number CE200100005).

## References

Breiman, L. (2001). Random forests. _Machine Learning_ , _45_(1), 5–32.
https://doi.org/10.1023/A:1010933404324

Friedman, J. H. (2001). Greedy function approximation: A gradient boosting
machine. _Annals of Statistics_ , 1189–1232.
https://doi.org/10.1214/aos/1013203451

Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015). Peeking inside
the black box: Visualizing statistical learning with plots of individual
conditional expectation. _Journal of Computational and Graphical Statistics_ ,
_24_(1), 44–65. https://doi.org/10.1080/10618600.2014.907095

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”:
Explaining the predictions of any classifier. _Proceedings of the 22nd ACM
SIGKDD International Conference on Knowledge Discovery and Data Mining_ ,
1135–1144. https://doi.org/10.1145/2939672.2939778

Rudin, C. (2019). Stop explaining black box machine learning models for high
stakes decisions and use interpretable models instead. _Nature Machine
Intelligence_ , _1_(5), 206–215. https://doi.org/10.1038/s42256-019-0048-x

Sokol, K., & Flach, P. (2020a). LIMEtree: Interactively customisable
explanations based on local surrogate multi-output regression trees. _arXiv
Preprint arXiv:2005.01427_. https://doi.org/10.48550/arXiv.2005.01427

Sokol, K., & Flach, P. (2020b). Towards faithful and meaningful interpretable
representations. _arXiv Preprint arXiv:2008.07007_.
https://doi.org/10.48550/arXiv.2008.07007

Sokol, K., & Flach, P. (2021a). Explainability is in the mind of the beholder:
Establishing the foundations of explainable artificial intelligence. _arXiv
Preprint arXiv:2112.14466_. https://doi.org/10.48550/arXiv.2112.14466

Sokol, K., & Flach, P. (2021b). You only write thrice: Creating documents,
computational notebooks and presentations from a single source. _Beyond Static
Papers: Rethinking How We Share Scientific Understanding in Machine Learning –
ICLR 2021 Workshop_. https://doi.org/10.48550/arXiv.2107.06639

Sokol, K., Hepburn, A., Poyiadzi, R., Clifford, M., Santos-Rodriguez, R., &
Flach, P. (2020). FAT Forensics: A Python toolbox for implementing and
deploying fairness, accountability and transparency algorithms in predictive
systems. _Journal of Open Source Software_ , _5_(49), 1904.
https://doi.org/10.21105/joss.01904

Sokol, K., Hepburn, A., Santos-Rodriguez, R., & Flach, P. (2019). bLIMEy:
Surrogate prediction explanations beyond LIME. _Workshop on Human-Centric
Machine Learning (HCML 2019) at the 33rd Conference on Neural Information
Processing Systems (NeurIPS)_. https://doi.org/10.48550/arXiv.1910.13016

Sokol, K., Santos-Rodriguez, R., & Flach, P. (2022). FAT Forensics: A Python
toolbox for algorithmic fairness, accountability and transparency. _Software
Impacts_ , _14_ , 100406. https://doi.org/10.1016/j.simpa.2022.100406

Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2018). mixup: Beyond
empirical risk minimization. _Proceedings of the 6th International Conference
on Learning Representations (ICLR)_. https://doi.org/10.48550/arXiv.1710.09412