# Revisiting Nancy Cartwright’s Notion of Reliability: Addressing Quantum
Devices’ Noise

Galina Weinstein  Reichman University, The Efi Arazi School of Computer
Science, Herzliya; University of Haifa, The Department of Philosophy, Haifa,
Israel.

###### Abstract

This paper serves as an addendum to my previously published work, which delves
into the experimentation with the Google Sycamore quantum processor under the
title ”Debating the Reliability and Robustness of the Learned Hamiltonian in
the Traversable Wormhole Experiment.” In the preceding publication, I
extensively discussed the quantum system functioning as a dual to a
traversable wormhole and the ongoing efforts to discover a sparse model that
accurately depicts the dynamics of this intriguing phenomenon. In this paper,
I bring to light an important insight regarding applying Nancy Cartwright’s
ideas about reliability and reproducibility, which are deeply rooted in
classical scientific practices and experiments. I show that when applied to
the realm of quantum devices, such as Google’s Sycamore quantum processor and
other Noisy Intermediate-Scale Quantum (NISQ) devices, these well-established
notions demand careful adaptation and consideration. These systems’ inherent
noise and quantum nature introduce complexities that necessitate rethinking
traditional perspectives on reliability and reproducibility. In light of these
complexities, I propose the term ”noisy reliability” as a means to effectively
capture the nuanced nature of assessing the reliability of quantum devices,
particularly in the presence of inherent quantum noise. This addendum seeks to
enrich the discussion by highlighting the challenges and implications of
assessing quantum device reliability, thereby contributing to a deeper
understanding of quantum experimentation and its potential applications in
various domains.

##  1 Introduction

This paper serves as an addendum to my previously published work [10], wherein
I conducted an in-depth exploration of the experimentation involving the
Google Sycamore quantum processor. In that earlier publication, I extensively
discussed the quantum system that functions as a dual to a traversable
wormhole and the dedicated endeavors to identify a sparse model that
accurately captures the dynamics inherent in such a phenomenon.

The preceding paper intricately explored the properties of the quantum system,
specifically focusing on the sparse model known as the learned Hamiltonian,
carefully crafted to simulate the dynamics of traversable wormholes
faithfully. The discussion in the previous publication encompassed a
comprehensive analysis of the sparse model, addressing its criticisms and
shedding light on the challenges arising from the background noise within the
Sycamore device. Furthermore, the perspective of reliability and robustness
was meticulously applied, critically examining the experiment’s fidelity in
confronting these challenges.

However, my journey remains incomplete without delving deeper into the
persistent quandary posed by noise in quantum devices. In this present
endeavor, I aim to explore the depths of this predicament, viewing it through
Nancy Cartwright’s philosophical lens of reliability.

Subsequently, I explore the intricate relationship between noise, reliability,
and the philosophical underpinnings that intertwine them. This paper is a
companion to my previous work, offering a more profound reflection on the
challenges and implications of noise and inviting contemplation on the
fundamental nature of reliability in the quantum domain.

In the realm of quantum systems, where inherent noise plays a crucial role in
the experimental setting, the conventional boundaries of reliability and
reproducibility become indistinct. Within this context, I navigate the nuances
of assessing the reliability of quantum devices, searching for a term that
encapsulates the essence of this dynamic interplay. The proposed term - ”noisy
reliability” - emerges, capturing the essence of my quest for a comprehensive
validation of experimentation within the realm of quantum noise. It
effectively reflects the acknowledgment of uncertainty in quantum mechanics
while seeking a new understanding of quantum device performance.

In the forthcoming sections, I delve into Cartwright’s thesis on the tangle of
science and reliability in Section 2, then explore the intrinsic relationship
between reliability and reproducibility in classical physics within Section 3.
As I venture further, Section 4 unravels the notion of reliability in the
context of quantum systems and devices, highlighting quantum noise’s
challenges. Ultimately, my journey culminates in section 5, in coining the
term ”noisy reliability” – a term that encapsulates the call for a new
perspective on validation in the realm of inherently noisy instruments.

##  2 Cartwright on Reliability

Cartwright argues that many scientific products we need to evaluate are not
focused on ”truth,” and therefore, we should shift our perspective to assess
their ” _in situ_ reliability.” Reliability refers to the ability of a
scientific theory or model to produce accurate predictions in a specific
context. Cartwright advocates for a shift in focus towards the reliability of
science’s panoply of outputs science produces, _the tangle of science_. This
shift entails moving from a general perspective to a more specific and
detailed examination.

Cartwright’s notion of the _tangle of science_ is an extensive and diverse
network comprising various components that can be utilized in diverse ways to
serve different purposes. It encompasses not only theoretical and experimental
elements but also models, concepts, measurement definitions, methods, bridging
principles, procedures, instruments, practices, concept development and
validation, case studies, coding, methods of approximation, methods of
inference, measures, evaluations, devices, statistical techniques, data
collection, analysis, curation, production, preservation, classification,
dissemination, non-experimental studies, narratives, plans, science-informed
policies, and numerous other products of scientific inquiry. Each of these
products carries equal importance in the scientific landscape. Particularly
significant is how these products interconnect and interweave, with their
collective synergy contributing to the success of any scientific endeavor [4],
[6].

In the context of the experiment discussed in my previous paper [10], the
approach advocated by Cartwright necessitates the evaluation of the
reliability of specific components within scientific practices. These
components include the quantum circuit and learned Hamiltonian, the
sparsification process (see [9]), and the reliability of the Sycamore quantum
processor.

The quantum circuit is composed of the following elements: researchers
prepared the thermofield double (TFD) state $\ket{\text{TFD}}$, which consists
of the eigenstates of the left and right quantum systems. They then created a
maximally entangled state between registers $P$ and $Q$. This step ensures
high entanglement between these two registers. Afterward, at the time
$t=t_{0}$, they applied a SWAP operation between registers $Q$ and $L$ to
insert the qubit into the wormhole, creating an entangled connection between
them. At the time $t=0$, they applied an interaction between the left and
right quantum systems, which is carried out between register $R$ and register
$L$. This interaction allows the qubit to interact with the left and right
quantum systems. Finally, at the time $t=t_{1}$, they applied another SWAP
operation between registers $R$ and $T$ to extract the qubit from the
wormhole, transferring the qubit’s entanglement from register $L$ to register
$T$. The mutual information $I_{PT}$ is measured between the registers $P$ and
$T$.111In the above quantum circuit protocol, $I_{PT}$ measures the
entanglement between register $P$, where a maximally entangled state was
prepared, and register $T$, where the qubit was extracted from the wormhole.
When we measure register $T$, the measurement outcome collapses the quantum
state of register $T$ to one of the basis states. This measurement outcome
will be correlated with the quantum state of register $P$ due to the prior
entanglement achieved between registers $P$ and $T$ through the interaction
and SWAP operations. After executing the protocol, the measured mutual
information $I_{PT}$ will quantitatively measure the entanglement between
these two registers [9], [11].

Cartwright emphasizes that by focusing on reliability rather than exclusively
on truth, we can gain deeper insights into the effectiveness of scientific
practices and their outputs. She suggests that the concern for reliability is
essential because truth, confirmation, and warrant are contingent upon it.
While she advocates for investigating the attributes that make these
scientific components reliable and trustworthy, she does not dismiss the
importance of truth altogether. However, she points out that assuming the
truth of a scientific claim without good reason and support for its
reliability puts us on shaky ground.

The shift in focus proposed by Cartwright encourages us to move away from
evaluating the truth of general principles and instead concentrate on the
reliability of various scientific products. Many of these products do not
apply to being labeled as ”true” or ”false.” Evaluating their reliability
becomes crucial, raising the question of what they are reliable for, which is
often overlooked. Overall, Cartwright’s advice calls for a comprehensive
assessment of the reliability of scientific outputs rather than solely seeking
truth. By focusing on reliability rather than exclusively on truth,
researchers can gain a deeper understanding of the effectiveness and
trustworthiness of the experiment’s results [4], [5], [6], [7], [8].

##  3 Reliability and Reproducibility

In her paper titled ”Replicability, Reproducibility, and Robustness,”
Cartwright asserts her lack of confidence in the reliability of instruments
when issues arise due to the implementation of a model rather than inherent
problems with the instruments themselves. Even if the instruments are highly
accurate in isolation, the process of implementing the model can magnify small
errors and uncertainties, resulting in less reliable overall predictions [2].

According to Cartwright, the reliability of scientific practices and their
outputs is considered essential because truth and confirmation rely on this
aspect. Suppose there is an issue with the reliability of the sparse model,
referred to as the learned Hamiltonian, which is designed to represent
traversable wormhole dynamics. In such a scenario, the implications extend to
the accuracy of the underlying model of a holographic semi-classical wormhole.
The learned Hamiltonian plays a pivotal role in the experiment, representing
the quantum system and governing its behavior. Hence, if the Hamiltonian is
unreliable or inaccurate, it leads to erroneous predictions and unreliable
outcomes within the experiment. I extensively addressed this in my prior paper
[10]. Therefore, the reliability of the learned Hamiltonian becomes critical,
as it significantly impacts the overall reliability of the experimental
results. The trustworthiness of the Hamiltonian has direct implications for
the validity of the underlying model and the conclusions drawn from the
experiment. Any uncertainties or errors in the Hamiltonian have the potential
to propagate throughout the experiment, casting doubt on the veracity of the
claims and principles based on its outcomes.

Cartwright emphasizes the importance of reliability for obtaining accurate and
trustworthy results, especially in cases where experiments cannot be easily
reproduced. She illustrates her point by referring to the Gravity Probe B
experiment, a substantial endeavor headed by Francis Everritt and a sizable
team of researchers.

Gravity Probe B, launched in 2004, is a space experiment utilizing cryogenic
gyroscopes in Earth’s orbit. In the Gravity Probe B project, the primary
objective was to observe and quantify a relativistic frame-dragging precession
phenomenon, which is a subtle effect predicted by Einstein’s general theory of
relativity. To achieve this high-precision measurement, the project team
constructed specialized gyroscopes that exhibit uniform and synchronous
rotation. Even slight variations in the manufacturing process of the
gyroscopes could adversely affect the precision of the measurements they aimed
to conduct. Therefore, the team opted for fused quartz as the gyroscope
material due to its ability to be uniformly produced. This ensured that all
components of the gyroscopes were identical, without any discrepancies.
Maintaining this uniformity is paramount because any variations in density
within the gyroscopes could lead to additional cyclic wobbling, which is
beyond control and prediction. This wobbling could subsequently hinder the
precise measurement of the relativistic effects, potentially obscuring their
accurate observation.

The project’s design and implementation took $45$ years due to the complexity
and importance of the observation. The reason for such an extended period is
to ensure that the design of the experiment is highly reliable and
trustworthy. However, uniformity alone did not suffice; just making the
gyroscopes uniform is insufficient. The experiment’s success hinged on the
team carefully considering potential external influences and disturbances that
could confound the measurements. Factors such as external forces and
interference were meticulously considered during the experimental design
phase. This encompassed strategies to minimize these factors’ impact and
enhance the visibility of the relativistic precession within the experimental
results.

It is noteworthy that the overarching goal of the Gravity Probe B project was
of significant magnitude. It sought to substantiate a fundamental aspect of
Einstein’s theory of relativity by investigating the interaction between a
rotating gyroscope and the curved fabric of space-time. The ultimate aim was
to experimentally validate Einstein’s theory’s predictions by examining
spinning gyroscopes [3]. However, as the team pursued their primary objective,
they stumbled upon an additional discovery that proved more practical and less
abstract. This newfound insight revolves around a general idea that can be
effortlessly articulated in a typical, straightforward manner. The concept
proposes that any gyroscope made of fused quartz, supported by magnets, coated
with a super-thin layer of superfluid, read using a special detector, kept at
cryogenic temperatures, and set into rotation in the depths of space, will
exhibit predictable behavior. Specifically, these gyroscopes will precess at a
specific rate that can be calculated and forecasted. Thus, while the primary
aim of the Gravity Probe B project is to test Einstein’s theory, it also
yielded valuable information about how gyroscopes with the mentioned specific
characteristics behave under particular conditions. This practical discovery
can be expressed as a general rule applicable to gyroscopes possessing these
unique attributes [3].

Cartwright expresses confidence in the instrument’s reliability, emphasizing
that being extremely certain about its accuracy is crucial since it will
likely provide a unique opportunity to gather data [2]. In 1989 Cartwright
wrote, ”The GP-B experiment is a good example. It may take twenty years, but
in the end, it should provide an entirely reliable test for the effects of
space-time curvature, a test as stringent as any empiricist could demand” [1].

##  4 Reliability of Quantum Devices

Now, I will apply Cartwright’s ideas to the field of noisy intermediate-scale
quantum (NISQ) computers such as Google’s Sycamore. The learned sparse SYK
model that retains critical attributes of the traversable wormhole (see [9])
was executed utilizing the Sycamore processor. A blog post on Google’s website
stated, ”The Google Sycamore processor is among the first to have the fidelity
needed to carry out this experiment” [12]. In other words, it is argued that
this high fidelity indicates that the Sycamore processor can maintain the
coherence and stability of quantum states throughout the computation, reducing
the impact of noise and errors. As a result, the Sycamore processor can
perform the wormhole experiment with great reliability and accuracy.

The inherent noise in quantum computations is a significant challenge in the
field of quantum computing and impacts the reliability of the Sycamore.
Quantum gates are susceptible to errors stemming from _decoherence_
(interactions with the external environment that introduce errors in quantum
gate operations), _gate crosstalk_ (where the operation of one gate affects
the intended operation of another nearby gate), and _imprecise calibration_ of
the quantum gates (that can result in inaccuracies during gate operations).
Additionally, quantum measurements can introduce noise and disturb the quantum
state (_readout errors_), affecting subsequent computations. Noise during the
measurement can introduce errors in the final results. Generally, quantum
systems inherently exhibit quantum noise due to the probabilistic nature of
quantum mechanics. This inherent noise can affect the accuracy of gate
operations. Sycamore’s noisy nature makes it difficult to express extreme
confidence in its reliability and accuracy, as small errors and uncertainties
can accumulate during computations. This can result in less reliable overall
outcomes, especially for complex calculations. In addition to the model
implementation potentially amplifying minor inaccuracies and leading to less
dependable predictions, it is crucial to acknowledge that the device is
susceptible to noise and errors. Noise and decoherence introduce uncertainties
that limit the reliability of the Sycamore processor.

As quantum technology advances and error-correction techniques improve, it is
expected that the reliability and accuracy of quantum computations will
improve over time. But it is important to acknowledge that evaluating the
reliability of the Sycamore quantum processor poses additional challenges due
to quantum sensitivity and noise, which raises doubts about the
trustworthiness of a single simulation run. Due to the presence of noise, we
cannot have complete confidence in the reliability of the processor.

In the case of Gravity Probe B and similar experiments involving classical
physical systems, noise can emerge from various sources, including
environmental factors, instrument imperfections, and external disturbances.
These noise sources can introduce uncertainties and deviations in the
measurements and outcomes of such experiments. These noise sources can affect
measurements’ reliability and reproducibility, but they are often more
predictable and manageable than the inherent noise present in quantum systems.
In Google’s Sycamore, noise is inherent. Quantum states are highly delicate
and can be easily disrupted by their surroundings, leading to errors in
quantum computations. This intrinsic noise poses significant challenges to
achieving reproducibility and reliability, as even small disturbances can lead
to unpredictable and uncontrollable effects on the quantum states being
manipulated.

Cartwright’s theory emphasizes the importance of reliability for confirming
scientific truths and obtaining accurate results. In the case of the Sycamore,
the challenges posed by quantum noise can hinder our ability to achieve the
same level of confidence and reliability as in macroscopic experiments like
Gravity Probe B. The inherent noise in NISQ devices introduces a level of
unpredictability that can make it difficult to obtain consistent outcomes
across multiple runs of an experiment, affecting the results’ reliability. In
other words, the inherent quantum noise is a fundamental challenge
distinguishing quantum systems from traditional macroscopic experiments like
Gravity Probe B.

##  5 Noisy reliability

The inherent noise in the Google Sycamore can hinder the ability to reproduce
experiments, which is a crucial aspect of scientific validation.
Reproducibility allows other researchers to verify experimental results and
build upon them independently. However, the inherent noise in the Sycamore
makes it challenging to reproduce the same computation and obtain consistent
outcomes. Reliability is intimately tied to reproducibility in scientific
experiments. The presence of noise and its challenges to reproducibility in
the context of the Sycamore can complicate assessing its reliability.

Suppose someone intends to execute the sparsified SYK model (which, as
asserted by its creators, maintains fundamental attributes of traversable
wormhole physics) on an alternate quantum device; the inherent noise within
the NISQ device might result in disparities during the experiment’s
simulation, potentially leading to distinct experimental outcomes. Reproducing
experiments across different quantum processors is not straightforward due to
inherent noise; different quantum processors have varying noise levels due to
differences in their qubit technologies and environmental conditions. Even if
two quantum processors have similar architectures and qubit counts, they may
still exhibit differences in performance and ability to reproduce the same
model precisely.

I argue that Cartwright’s notion of reliability needs to be adapted when
applied to quantum instruments, particularly quantum processors. The inherent
quantum nature and noise in quantum systems introduce complexities that may
require rethinking traditional notions of reliability. Assessing the
reliability of quantum devices, such as quantum processors, involves
considering the impact of quantum noise and understanding the interplay
between quantum states, measurements, and computations. This calls for a more
nuanced perspective on reliability in the context of quantum computing.

In quantum computing, noisy simulating is the process of simulating a quantum
system in the presence of noise. These simulations involve incorporating noise
models into the algorithms and circuits being simulated. In a similar vein, I
propose the term ”noisy reliability.” I suggest that the term ”noisy
reliability” effectively captures the nuanced nature of assessing the
reliability of quantum devices, especially in inevitable quantum noise. This
term reflects the understanding that while perfect reproducibility may be
challenging to achieve in the noisy quantum realm, researchers can still work
to establish a realistic assessment of the reliability and validity of the
outcomes. ”Noisy reliability” acknowledges that noise is an inherent part of
quantum systems and that researchers must navigate this uncertainty while
striving to extract meaningful and accurate information from quantum
experiments. It signifies a pragmatic approach that considers the limitations
imposed by quantum noise while still aiming to obtain reliable and valuable
results, even though quantum systems may not always produce the same results
upon repeated measurements due to inherent quantum noise.

In the field of quantum computing, where noise is a fundamental challenge, the
concept of ”noisy reliability” captures the essence of how researchers work to
understand, characterize, and mitigate noise effects to achieve trustworthy
and informative outcomes. This term underscores the importance of
acknowledging and addressing noise while still striving to establish a
reliability level appropriate within the context of quantum systems.

## Acknowledgement

This work is supported by ERC advanced grant number 834735.

## References

  * [1] N. Cartwright (1989). _Nature’s Capacities and their Measurement._ Oxford: Oxford University Press.
  * [2] N. Cartwright (1991). ”Replicability, Reproducibility, and Robustness: Comments on Harry Collins.” _History of Political Economy_ 23, pp. 143-155.
  * [3] N. Cartwright (1999). _The Dappled World. A Study of the Boundaries of Science._ Oxford: Oxford University Press.
  * [4] N. Cartwright (2020). ”X - Why Trust Science? Reliability, Particularity and the Tangle of Science.” _Proceedings of the Aristotelian Society_ cxx, pp. pp. 237-252.
  * [5] N. Cartwright (2022). _A Philosopher Looks at Science._ Cambridge: Cambridge University Press
  * [6] N. Cartwright and J. Hardie and E. Montuschi and S. Soleiman and A. C. Thresher (2022). _The Tangle of Science: Reliability Beyond Method, Rigour, and Objectivity._ Oxford: Oxford University Press.
  * [7] N. Cartwright (2023). ”Science In the Earthly Plane.” _University of Amsterdam. Institute for Advanced Study (IAS)_.
  * [8] P. Galison, J. Doboszewski, J. Elder, N. C. M. Martens, et.al. (2023). ”The Next Generation Event Horizon Telescope Collaboration: History, Philosophy, and Culture.” _Galaxies_ 11, pp. 1-23.
  * [9] D. Jafferis, A. Zlokapa, J. D. Lykken, D. K. Kolchmeyer, S. I. Davis, N. Lauk, H. Neven, and M. Spiropulu (2022). ”Traversable wormhole dynamics on a quantum processor.” _Nature_ 612, pp. 51–55.
  * [10] G. Weinstein (2023). ”Debating the Reliability and Robustness of the Learned Hamiltonian in the Traversable Wormhole Experiment.” _arXiv_ :2308.00697v1.
  * [11] Zlokapa, A. (2021). _Quantum Computing for Machine Learning and Physics Simulation._ Thesis. Caltech.
  * [12] Zlokapa, A. (2022). ”Making a Traversable Wormhole with a Quantum Computer.” _Google Research_.


# Knowledge Distillation for Multi-Target Domain Adaptation in Real-Time
Person Re-Identification

###### Abstract

Despite the recent success of deep learning architectures, person re-
identification (ReID) remains a challenging problem in real-word applications.
Several unsupervised single-target domain adaptation (STDA) methods have
recently been proposed to limit the decline in ReID accuracy caused by the
domain shift that typically occurs between source and target video data. Given
the multimodal nature of person ReID data (due to variations across camera
viewpoints and capture conditions), training a common CNN backbone to address
domain shifts across multiple target domains, can provide an efficient
solution for real-time ReID applications. Although multi-target domain
adaptation (MTDA) has not been widely addressed in the ReID literature, a
straightforward approach consists in blending different target datasets, and
performing STDA on the mixture to train a common CNN. However, this approach
may lead to poor generalization, especially when blending a growing number of
distinct target domains to train a smaller CNN. To alleviate this problem, we
introduce a new MTDA method based on knowledge distillation (KD-ReID) that is
suitable for real-time person ReID applications. Our method adapts a common
lightweight student backbone CNN over the target domains by alternatively
distilling from multiple specialized teacher CNNs, each one adapted on data
from a specific target domain. Extensive experiments111GitHub code:
https://github.com/fremigereau/MTDA_KD_REID. conducted on several challenging
person ReID datasets indicate that our approach outperforms state-of-art
methods for MTDA, including blending methods, particularly when training a
compact CNN backbone like OSNet. Results suggest that our flexible MTDA
approach can be employed to design cost-effective ReID systems for real-time
video surveillance applications.

Index Terms—  Video Surveillance, Person Re-Identification, Multi-Target
Domain Adaptation, Knowledge Distillation.

##  1 Introduction

Person ReID aims to recognize an individual captured over a set of non-
overlapping camera viewpoints. It is an important function required in many
computer vision applications, ranging from sports analytics to video
surveillance [1]. State-of-the-art approaches for person ReID are typically
implemented with a deep learning (DL) model, e.g., deep Siamese networks,
trained through metric learning to provide global appearance features. CNN
backbones are often used to learn an embedding, where similar image pairs
(with the same identity) are close to each other, and dissimilar image pairs
(with different identities) are distant from each other. Popular supervised
losses include cross-entropy, triplet, and contrastive losses based on
positives and negatives pairs in a labeled dataset [2]. Despite their success,
they remain complex models, and typically require optimizing many parameters
using large annotated image datasets. They also suffer from poor
generalization in the presence of domain shift, where the distribution of
original video data captures from the source domain diverges w.r.t data from
the operational target domain. Domain shifts are introduced by variations in
capture conditions (i.e., pose and illumination), camera settings and
viewpoints from which a person is observed, and lead to considerable changes
in the distribution of images in different datasets [3, 4].

Given the cost of data annotation, several unsupervised STDA methods have
recently been proposed to limit the decline of ReID accuracy caused by domain
shift. These methods seek to adapt CNNs trained with annotated source video
data to perform well in a target domain by leveraging unlabeled data captured
from that domain. To learn a discriminant domain-invariant feature
representation from source and target data, STDA methods rely on, e.g.,
discrepancy-based or adversarial approaches [5, 3, 6, 7, 8, 9, 10, 11, 12,
13].

Although MTDA based on unlabeled data from multiple targets is important in
many real-world applications, it remains largely unexplored [14]. Existing
STDA techniques can be extended to MTDA by either adapting multiple models,
one for each target domain, or by blending data from the multiple target
domains, and then applying STDA on the mixture of target data. For image
classification, an adversarial MTDA approach based on blending transfer was
recently proposed where all targets are viewed as one domain [15], target-
specific representations were concatenated to deploy a common model [16].
Nevertheless, these approaches are either too complex it requires one model
per target domain, or generalize poorly on distinct target domains,
particularly when adapting a smaller common CNN backbone on a growing number
of targets. In [6], MTDA is performed by distilling information from target-
specific teachers to a student model (deployed for testing), significantly
reducing system complexity.

In person ReID, video cameras in a distributed network may correspond to
different target domains (defined by viewpoints and capture conditions), and
the CNN backbone should therefore be adapted to generalize well across
multiple target domains. More recently, Tian et al. [17] introduced an MTDA
method for ReID. Inspired by the importance of intra-domain camera style, they
propose a Camera Identity-guided Distribution Consistency method based on
distribution consistency and discriminative embedding losses. The first loss
aligns image style of source and targets through a generative approach, while
the second predicts the camera for each identity to decrease distances for
each individual across cameras. However, for real-time person ReID,
specialized MTDA techniques are required to train smaller cost-effective CNN
backbones that can address domain shifts across multiple data distributions.
In this paper, we advocate for MTDA methods based on knowledge distillation
(KD) [18, 14] to provide a better trade-off between CNN accuracy and
efficiency.

In this paper, we propose KD-ReID, a cost-effective MTDA method for real-time
person ReID applications. A CNN backbone (teacher) is adapted for each target
domain, using an STDA method, and the corresponding unlabeled target dataset.
Then, the knowledge learned from all the target-specific teachers is
alternatively distilled to a common lightweight CNN backbone (student). This
paper expands significantly from the KD method in [14], which was proposed for
image classification tasks on larger CNNs. Additionally, KD-ReID differs
considerably from a KD method for person ReID that was proposed in [19] for
multi-source domain adaptation (MSDA). It was applied in a semi-supervised
learning scenario and distills knowledge from multiple source models
(teachers) to a target model (student) by aligning student-teacher similarity
matrices. The accuracy and complexity of KD-ReID are compared extensively
against state-of-the-art MTDA methods on several challenging person ReID
datasets.

##  2 Proposed Approach

Fig. 1: Overview of MTDA methods. (a) Blending: target domain datasets are
combined to form a dataset, and the common CNN is adapted using a STDA method.
(b) KD-ReID (ours): individual teachers are adapted using a STDA method for
each target, and knowledge is distilled into the common CNN.

Let $\mathbf{x}^{s}\in\mathcal{X}^{s}$ be the set of samples from the source
domain, and ${y}^{s}\in\mathcal{Y}$ be their corresponding labels, where
$\mathcal{X}^{s}$ is the source domain feature space, and $\mathcal{Y}$ is the
source label space. Given the target domains feature space $\mathcal{X}^{t}$,
let
$\mathbf{x}^{t}=\left\\{\mathbf{x}_{1}^{t},\mathbf{x}_{2}^{t},...,\mathbf{x}_{T}^{t}\right\\}\in\mathcal{X}^{t}$
be the set of samples for $T$ unlabeled target domain datasets. We define the
common CNN backbone as $\Theta$, and a set of target-specific CNN backbones,
each one adapted to each target domain, as
$\mathbf{\Phi}=\\{\Phi_{1},\Phi_{2},...,\Phi_{T}\\}$. As illustrated in Figure
1 (a), a straightforward MTDA method for adapting a common CNN backbone
$\Theta$ consists in blending the data $\mathbf{x}_{i}^{t}$ of all of the
targets for $i=1,2,...,T$, and then applying a STDA method on the resulting
dataset $\mathbf{x}^{t}$.

We introduce KD-ReID, a MTDA method based on KD that is suitable for real-time
person ReID. Our method adapts a common lightweight CNN backbone (student
model) $\Theta$ to address multiple domain shifts, and perform well over all
the target domains. As illustrated in Figure 1 (b), our proposed method
distills knowledge learned by multiple specialized CNN backbones $\Phi_{i}$
(teacher models), each one adapted on data $\mathbf{x}_{i}^{t}$ from a
specific target domain, for $i=1,2,...,T$, using a STDA method. In particular,
the KD-ReID method consists of three steps. (1) Pre-training on source data:
All the CNN backbones (teachers $\Phi_{i}$ and student $\Theta$ models)
undergo supervised pre-training on the labeled source domain dataset
$\mathbf{x}^{s}$. This process should provide discriminant networks for person
ReID on the source data. Any appropriate supervised loss functions for metric
learning can be applied, including softmax cross-entropy [20], and hard
samples mining triplet loss [21]. (2) STDA of teacher CNNs: The teacher models
$\Phi_{i}$ are adapted to their respective target domains using some STDA
method with source $\mathbf{x}^{s}$ and target $\mathbf{x}_{i}^{t}$ datasets,
for $i=1,2,...,T$. Any STDA method can be applied, including the D-MMD [3],
and SPCL [22] methods. (3) KD to student CNN: Knowledge from every teacher
model $\Phi_{i}$ is distilled to the common student model $\Theta$ based on
target datasets $\mathbf{x}_{i}^{t}$, for $i=1,2,...,T$, and on the KD loss
function, $\mathcal{L}_{KD}$ (described below). The KD-ReID method is
summarized in Algorithm 1.

labeled source datasets: $\mathbf{x}^{s}$, unlabeled target data:
$\mathbf{x}^{t}=\left\\{\mathbf{x}_{1}^{t},\mathbf{x}_{2}^{t},...,\mathbf{x}_{T}^{t}\right\\}$,

1) Pre-train teacher models,
$\mathbf{\Phi}=\\{\Phi_{1},\Phi_{2},...,\Phi_{T}\\}$, and student model,
$\Theta$, on the source labeled data $\mathbf{x}^{s}$

for $i=1,2,...T$ do

for each mini-batches $B^{s}\subset\mathbf{x}^{s}$ and
$B^{t}\subset\mathbf{x}^{t}_{i}$ do

2) STDA: adapt teacher $\Phi_{i}$ on source and target data

end for

end for

for $i=1,2,...T$ do

for each mini-batch $B^{t}\subset\mathbf{x}_{i}^{t}$ do

3) KD: adapt student $\Theta$ using target data and $\mathcal{L}_{KD}$

end for

end for

Algorithm 1 KD-ReID method.

Once the set of teacher models $\mathbf{\Phi}$ are adapted to respective
target domains, KD is proposed to progressively integrate their knowledge into
the common student model $\Theta$. Based on the self-similarity matrices of
feature vectors from $\mathbf{\Phi}$ and $\Theta$, our proposed KD loss
$\mathcal{L}_{KD}$ aligns student-teacher representations. Let
$f_{n}=\Phi_{i}(x_{n})$ be the feature vector output from the fully connected
layer of $\Phi_{i}$ in response to sample $x_{n}$. Consider a feature matrix
$\mathbf{F}=[f_{1},f_{2},...,f_{N}]\in\mathbb{R}^{D\times N}$ where $N$ is the
batch size and $D$ is the dimension of feature vectors. The self-similarity
matrix denoted as $\mathbf{A}\in\mathbb{R}^{N\times N}$ is defined as
$a_{j,k}=\langle f_{j},f_{k}\rangle$, where $\langle\cdot,\cdot\rangle$ is the
cosine distance between the feature vectors and $f_{j}$, $f_{k}\in\mathbf{F}$
. Similarity matrices $\mathbf{A}^{s}$ and $\mathbf{A}^{t}$ are computed based
on student $\mathbf{F}^{s}$ and teacher $\mathbf{F}^{t}$ feature matrices,
respectively. Our KD loss $\mathcal{L}_{KD}$ is then computed as:
$\mathcal{L}_{KD}=||\mathbf{A}^{s}-\mathbf{A}^{t}||_{F}$, where
$||\cdot,\cdot||_{F}$ is the Frobenius norm matrix. Optimizing $\Theta$
parameters with $\mathcal{L}_{KD}$ allows the student model to produce a
distance matrix aligned to the teacher model. We only apply this loss on
target samples $\mathbf{x}^{t}$. It allows tuning $\Theta$ parameters, while
$\mathbf{\Phi}$ parameters are fixed. For each mini-batch $B^{t}$ from a given
target domain $i$, we select only the corresponding teacher model $\Phi_{i}$.
The order of targets for KD is selected randomly, and it changes at every
epoch.

An important benefit of the KD-ReID approach is its versatility. It is
independent of the STDA methods used to adapt each $\Phi_{i}$. The choice of
CNN backbones $\Phi_{i}$, and its STDA methods may differ, allowing to select
the best configuration to address the specific challenges (i.e., domain shift)
faced by each target domain. Moreover, KD-ReID allows distilling knowledge to
a lighter common CNN backbone (student model), making it suitable for real-
time applications.

##  3 Experimental Results

MTDA Method – base STDA | Accuracy on Target Data (%) | Complexity
---|---|---
Market1501 | DukeMTMC | CUHK03 | Average
mAP | R1 | mAP | R1 | mAP | R1 | mAP | R1 | # Parameters | FLOPs
Lower Bound: Sup. on Source | 24.9 | 50.8 | 27.9 | 47.0 | 25.6 | 29.1 | 26.1 | 42.3 | 0.4 M | 0.08 G
One Model / Target – D-MMD∗ | 51.4 | 74.9 | 51.4 | 69.3 | 61.8 | 65.9 | 54.9 | 70.0 |  $T$ x 27.7 M | 2.70 G
Blending Targets – D-MMD | 32.5 | 59.5 | 35.5 | 55.6 | 35.8 | 40.5 | 34.6 | 51.9 | 0.4 M | 0.08 G
KD-ReID – D-MMD (Ours) | 46.7 | 70.9 | 47.4 | 66.8 | 51.0 | 55.3 | 48.4 | 64.3 | 0.4 M | 0.08 G
One Model / Target – SPCL∗ | 54.2 | 75.3 | 52.0 | 69.6 | 33.4 | 34.8 | 45.9 | 59.9 |  $T$ x 27.7 M | 2.70 G
Blending Targets – SPCL | 31.5 | 54.4 | 31.5 | 48.9 | 16.4 | 12.1 | 26.5 | 38.5 | 0.4 M | 0.08 G
KD-ReID – SPCL (Ours) | 51.5 | 74.6 | 45.3 | 64.7 | 37.1 | 39.8 | 44.6 | 59.7 | 0.4 M | 0.08 G
KD-ReID – Mixed (Ours) | 53.3 | 75.4 | 48.3 | 67.0 | 50.2 | 54.4 | 50.6 | 65.6 | 0.4 M | 0.08 G
Upper Bound: Sup. on Targets | 62.3 | 82.7 | 58.3 | 74.8 | 62.6 | 64.8 | 61.1 | 74.1 | 0.4 M | 0.08 G

Table 1: Performance of MTDA methods when MSMT17 is used as the source
dataset, and Market1501, DukeMTMC, and CUHK03 are used as target datasets
($T=3$ targets), with 2 STDA techniques – D-MMD and SPCL. Resnet50 is used to
implement target-specific CNN backbones, and Osnet_x0_25 as the architecture
for the common CNN backbones. The lower bound performance is obtained through
supervised training of Osnet_x0_25 on the labeled source dataset only, and
then evaluation on the 3 target datasets. For the upper bound, supervised
training of Osnet_x0_25 is performed on the blended target datasets. "*": For
the "One Model / Target" methods, the Resnet50 (target-specific) performance
is presented as an upper bound that should not be directly compared with other
MTDA techniques. FLOPs are related to the extraction features for one sample.

Datasets: In this paper, we evaluate MTDA methods across datasets, where each
dataset is a target domain. Four benchmark datasets are used for person ReID
experiments. CUHK03[23] consists of 13,164 images of 1,467 identities from 5
camera pairs. Market1501 [24] is comprised of a high number of images per
identities (32,217 images of 1,501 identities) captured with 6 cameras.
Compared to CUHK03, the bounding box annotations in Market1501 are lower
quality, and persons captured in each bounding box could be partially
represented, which makes for a challenging dataset. DukeMTMC[25] is designed
such that each one of the 8 cameras contains unique identities. This dataset
contains 36,441 images of 1,812 identities, which is similar in size to
Market1501 in terms of the number of images and identities. Finally, MSMT17
[26] includes video from 15 indoor and outdoor cameras, and consists of
126,441 images of 4,101 identities. Considering that videos are captured in
indoor/outdoor conditions, during different days, weather conditions, and
times of the day, it certainly represents a more challenging and realistic
dataset in comparison to other person ReID datasets. MSMT17 is always used as
the source dataset to pre-train the CNNs. Considering the diversity of capture
conditions, there is a significant domain shift between samples from these
datasets (results not shown in this paper).

Experimental Setting: Resnet50 [27] is used to implement the target-specific
CNN backbones (teachers), while Osnet_x0_25 [28] implements the common CNN
backbone (deployed for testing). A fully connected layer with 2048 neurons is
added to each backbone to output feature vectors. The original hyperparameter
setup for D-MMD [3] and SPCL [22] STDA techniques are used for our
experiments. For KD-ReID, the Stochastic Gradient Descent (SGD) optimizer is
used with a learning rate of 0.01. It is divided by 10 after every five
epochs. A batch size of 64 is used, with 32 samples from the source data, and
32 samples from the target data. Batches are constructed using groups of four
images with the same identity corresponding to tracklets. Training is
conducted until the model performance improves by less than 0.5% average mAP
over five consecutive epochs.

Results and Discussion: Table 1 compares the performance of the proposed KD-
ReID to the baseline methods in terms of mean Average Precision (mAP), rank-1
accuracy, number of parameters, and FLOPs. As expected, the lower-bound
accuracy on target datasets is low due to the domain shift between the source
and target datasets. Blending all the target data, and then applying an STDA
method improves accuracy significantly over the lower bound. The proposed KD-
ReID outperforms the blending methods on all datasets for both D-MMD and SPCL
STDA approaches. Although training a single model per target is a
straightforward approach that achieves a higher level of accuracy, at the
expense of high complexity. As shown in Table 1, the SPCL technique for STDA
produces a less accurate teacher model for the CUHK03 dataset than D-MMD. This
observation motivates us to select the best STDA technique for each teacher
model. The highest average performance is obtained when mixing STDA methods to
adapt teachers for KD-ReID, highlighting its versatility.

Table 2 shows the performance of KD-ReID versus two related SOTA methods.
First, Wu et. al. [18] proposed an MSDA method based on KD for a semi-
supervised scenario. The method should be robust to domain shift across new
domains since it benefits from some labeled target domain data, and the
labeled data from multiple different source domains to adapt the common CNN
backbone. In practice, this scenario is restrictive for ReID, given the need
to collect datasets and annotate samples. Yet, our KD-ReID outperforms this
method significantly (e.g., by 21.1 % in mAP, and 15.1% in rank-1 on
Market1501), even with a lighter CNN backbone. Second, Tian et. al. [17]
recently proposed the only MTDA method in literature for ReID. Our method
significantly outperforms this approach with an improvement of 18.3% in mAP,
and 6.2% in rank-1 on average, even on a much smaller CNN architecture (by a
ratio close to 60). In contrast, our KD-ReID method divides the problem into
simpler STDA problems, that may be solved using any combination of suitable
SOTA methods, rather than resolving the complex MTDA problem. The compression
with KD allows for a lightweight student model to reproduce the performance of
the highly optimized teachers.

MTDA Method – CNN Backbone | Accuracy on Target Data (%) | Complexity
---|---|---
Market1501 | DukeMTMC | Average
mAP | R1 | mAP | R1 | mAP | R1 | # Para. | FLOPs
Wu [18] – MobileNetV2 | 33.5 | 61.5 | 29.4 | 48.4 | 31.5 | 55.0 |  $T$ x 4.3 M | 0.40 G
Tian [17] – Resnet50 | 35.9 | 70.4 | 33.6 | 57.2 | 34.8 | 63.8 | 23.5 M | 2.70 G
| KD-ReID: SPCL Teachers
---
– OSNet_x0_25 (ours)
54.2 | 76.6 | 47.8 | 66.2 | 51.0 | 71.4 | 0.4 M | 0.08 G

Table 2: Performance of KD-ReID versus related SOTA methods on 2 target
domains MSMT17 $\rightarrow$ Market1501 & DukeMTMC. We consider multiple
models for [18] since it is a MSDA method, where $T$ is the number of targets.

Figure 2 compares the average performance of the common backbone CNN obtained
using the KD-ReID and blending methods. The experiments are conducted using
three variations of the Osnet CNN: osnet_x1_0, osnet_x0_5, and osnet_x0_25,
having about 4.0M, 1.0M, and 0.4M parameters, respectively. As shown in the
figure, the gap in accuracy between the two methods widens as the model
complexity is reduced. KD-ReID always outperforms the blending technique and
remains significantly more effective for adapting lightweight models, thus
more suitable for real-time applications.

Fig. 2: Impact of student model complexity on the average mAP over targets
(Market1501, DukeMTMC, and CUHK03), and using MSMT17 as source. The common CNN
is Osnet, while teachers are Resnet50s, and D-MMD is used for STDA.

##  4 CONCLUSION

In this paper, KD-ReID is introduced as a versatile MTDA method for real-time
person ReID. To adapt each teachers model, KD-ReID allows selecting the STDA
method and CNN individually, and then knowledge of all teacher models is
distilled to a common lightweight CNN (student). Our experiments conducted on
challenging person ReID datasets indicate that student models adapted through
KD-ReID outperform blending methods, and generalize well on all targets at the
same time, even when adapting a small common CNN backbone on a growing number
of targets. Such scalability paves the way for cost-effective systems for
real-time applications.

## References

  * [1] Y. He, X. Wei, X. Hong, W. Shi, and Y. Gong,  “Multi-target multi-camera tracking by tracklet-to-target assignment,”  IEEE Trans. Image Processing, vol. 29, pp. 5191–5205, 2020.
  * [2] M. Boudiaf, J. Rony, I.M. Ziko, E. Granger, M. Pedersoli, P. Piantanida, and I. Ben Ayed,  “A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses,”  in ECCV 2020.
  * [3] D. Mekhazni, A. Bhuiyan, G. Ekladious, and E. Granger,  “Unsupervised DA in the dissimilarity space for person reID,”  in ECCV 2020.
  * [4] S. Xuan and S. Zhang,  “Intra-inter camera similarity for unsupervised person reID,”  arXiv 2021.
  * [5] L. Wang and K Yoon,  “Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks,”  IEEE Trans. Pattern Analysis and Machine Intelligence, 2021.
  * [6] L.T. Nguyen-Meidine, A. Belal, M. Kiran, J. Dolz, L. A. Blais-Morin, and E. Granger,  “Knowledge distillation methods for efficient unsupervised adaptation across multiple domains,”  Image and Vision Computing, vol. 108, 2021.
  * [7] L.T. Nguyen-Meidine, E. Granger, M. Kiran, J. Dolz, and L.A. Blais-Morin,  “Joint progressive knowledge distillation and unsupervised DA,”  arXiv 2020.
  * [8] Z. Liu, Z. Miao, X. Pan, X. Zhan, D. Lin, S. Yu, and B. Gong,  “Open compound DA,”  in CVPR 2020.
  * [9] G. Ekladious, H. Lemoine, E. Granger, K. Kamali, and S. Moudache,  “Dual-triplet metric learning for unsupervised DA in video-based face recognition,”  arXiv 2020.
  * [10] L.T. Nguyen-Meidine, M. Kiran, M. Pedersoli, J. Dolz, L.A. Blais-Morin, and E. Granger,  “Incremental multi-target DA for object detection with efficient domain transfer,”  arXiv 2021.
  * [11] Y. Ge, Z. Li, H. Zhao, G. Yin, S. Yi, X. Wang, and H. Li,  “Fd-gan: Pose-guided feature distilling gan for robust person reID,”  in NeurIPS 2018.
  * [12] S. Khurana, N. Moritz, T. Hori, and J. Le Roux,  “Unsupervised domain adaptation for speech recognition via uncertainty driven self-training,”  in ICASSP 2021.
  * [13] Z. Allen-Zhu and Y. Li,  “Towards understanding ensemble, knowledge distillation and self-distillation in deep learning,”  arXiv 2020.
  * [14] L.T. Nguyen-Meidine, A. Belal, M. Kiran, J. Dolz, L.A. Blais-Morin, and E. Granger,  “Unsupervised multi-target DA through knowledge distillation,”  in WACV 2021.
  * [15] Z. Chen, J. Zhuang, X. Liang, and L. Lin,  “Blending-target domain adaptation by adversarial meta-adaptation networks,”  in CVPR 2019.
  * [16] B. Gholami, P. Sahu, O. Rudovic, K. Bousmalis, and V. Pavlovic,  “Unsupervised multi-target domain adaptation: An information theoretic approach,”  IEEE Trans. Image Processing, vol. 29, pp. 3993–4002, 2020.
  * [17] J. Tian, Q. Tang, R. Li, Z. Teng, B. Zhang, and J. Fan,  “A camera identity-guided distribution consistency method for unsupervised multi-target domain person reID,”  ACM Trans. Intelligent Systems and Technology, vol. 12, no. 4, pp. 1–18, 2021.
  * [18] A. Wu, W.S. Zheng, X. Guo, and J.H. Lai,  “Distilled person reID : Towards a more scalable system,”  in CVPR 2019.
  * [19] A. Wu, W.S. Zheng, X. Guo, and J.H. Lai,  “Distilled person reID : Towards a more scalable system,”  in CVPR 2019.
  * [20] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna,  “Rethinking the inception architecture for computer vision,”  in ICCV 2016.
  * [21] A. Hermans, L. Beyer, and B. Leibe,  “In defense of the triplet loss for person reID,”  arXiv 2017.
  * [22] Y. Ge, F. Zhu, D. Chen, R. Zhao, et al.,  “Self-paced contrastive learning with hybrid memory for domain adaptive object re-id,”  NeurIPS 2020.
  * [23] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang,  “Deepreid: Deep filter pairing neural network for person reID,”  in CVPR 2014.
  * [24] L. Zheng, L. Shen, L. Tian, S. Wang, J. Wang, and Q. Tian,  “Scalable person reID : A benchmark,”  in ICCV 2015.
  * [25] Z. Zhang, J. Wu, X. Zhang, and C. Zhang,  “Multi-target, multi-camera tracking by hierarchical clustering: Recent progress on dukemtmc project,”  arXiv 2017.
  * [26] L. Wei, S. Zhang, W. Gao, and Q. Tian,  “Person transfer gan to bridge domain gap for person reID,”  in CVPR 2018.
  * [27] K. He, X. Zhang, S. Ren, and J. Sun,  “Deep residual learning for image recognition,”  in CVPR 2016.
  * [28] K. Zhou, Y. Yang, A. Cavallaro, and T. Xiang,  “Omni-scale feature learning for person reID,”  in ICCV 2019.