# Be Aware of the Neighborhood Effect: Modeling Selection Bias under
Interference

Haoxuan Li1 Chunyuan Zheng1 Sihao Ding2 Peng Wu3, Zhi Geng3
Fuli Feng2 Xiangnan He2
1Peking University 2University of Science and Technology of China
3Beijing Technology and Business University
<EMAIL_ADDRESS><EMAIL_ADDRESS>
{zhengchunyuan99, fulifeng93<EMAIL_ADDRESS>
{pengwu<EMAIL_ADDRESS>Corresponding author.

###### Abstract

Selection bias in recommender system arises from the recommendation process of
system filtering and the interactive process of user selection. Many previous
studies have focused on addressing selection bias to achieve unbiased learning
of the prediction model, but ignore the fact that potential outcomes for a
given user-item pair may vary with the treatments assigned to other user-item
pairs, named neighborhood effect. To fill the gap, this paper formally
formulates the neighborhood effect as an interference problem from the
perspective of causal inference and introduces a treatment representation to
capture the neighborhood effect. On this basis, we propose a novel ideal loss
that can be used to deal with selection bias in the presence of neighborhood
effect. We further develop two new estimators for estimating the proposed
ideal loss. We theoretically establish the connection between the proposed and
previous debiasing methods ignoring the neighborhood effect, showing that the
proposed methods can achieve unbiased learning when both selection bias and
neighborhood effect are present, while the existing methods are biased.
Extensive semi-synthetic and real-world experiments are conducted to
demonstrate the effectiveness of the proposed methods.

##  1 Introduction

Selection bias is widespread in recommender system (RS) and challenges the
prediction of users’ true preferences (Wu et al., 2022; Chen et al., 2023),
which arises from the recommendation process of system filtering and the
interactive process of user selection (Marlin and Zemel, 2009; Huang et al.,
2022). For example, in the rating prediction task, selection bias happens in
explicit feedback data as users are free to choose which items to rate, so
that the observed ratings are not a representative sample of all ratings
(Steck, 2010; Wang et al., 2023c). In the post-click conversion rate (CVR)
prediction task, selection bias happens due to conventional CVR models are
trained with samples of clicked impressions while utilized to make inference
on the entire space with samples of all impressions (Ma et al., 2018; Zhang et
al., 2020; Wang et al., 2022a; Li et al., 2023f).

Inspired by the causal inference literature (Imbens and Rubin, 2015), many
studies have proposed unbiased estimators for eliminating the selection bias,
such as inverse propensity scoring (IPS) (Schnabel et al., 2016), self-
normalized IPS (SNIPS) (Swaminathan and Joachims, 2015), and doubly robust
(DR) methods (Wang et al., 2019; Chen et al., 2021a; Dai et al., 2022; Li et
al., 2023d; e). Given the features of a user-item pair, these methods first
estimate the probability of observing that user rating or clicking on the
item, called propensity. Then the inverse of the propensity is used to weight
the observed samples to achieve unbiased estimates of the ideal loss.

However, the theoretical guarantees of the previous methods are all
established under the Stable Unit Treatment Values Assumption (SUTVA) (Rubin,
1980), which requires that the potential outcomes for one user-item pair do
not vary with the treatments assigned to other user-item pairs (also known as
no interference or no neighborhood effect), as shown in Figure 1(a). In fact,
such an assumption can hardly be satisfied in real-world scenarios. For
example, a user’s rating on an item can be easily influenced by other users’
ratings on that item, as well as a user’s clicking on an item might facilitate
other users’ clicking and purchasing of that item (Chen et al., 2021b; Zheng
et al., 2021). Figure 1(b) shows a general causal diagram in the presence of
interference in debiased recommendation.

Figure 1: Causal diagrams of the existing debiasing methods under no
interference assumption (left), and the proposed method taking into account
the presence of interference (right), where $x_{u,i}$, $o_{u,i}$, and
$r_{u,i}$ denote the confounder, treatment, and outcome of user-item pair
$(u,i)$, respectively. In the presence of interference, $\mathcal{N}_{(u,i)}$
and $\mathcal{N}_{-(u,i)}$ denote the other user-item pairs affecting and not
affecting $(u,i)$, respectively, and $\bm{g}_{u,i}$ denotes the treatment
representation to capture the interference.

To fill this gap, in this paper, we first formulate the debias problem in
Figure 1(b) from the perspective of causal inference and extend the definition
of potential outcomes to be compatible in the presence of interference, then
introduce a learnable treatment representation to capture such interference.
Based on the extended potential outcome and treatment representation, we
propose a novel ideal loss that can effectively evaluate the performance of
the prediction model when both selection bias and neighborhood effect are
present. We then propose two new estimators for estimating the proposed ideal
loss, named neighborhood inverse propensity score (N-IPS) and neighborhood
doubly robust (N-DR), respectively. Theoretical analysis shows that the
proposed N-IPS and N-DR estimators can achieve unbiased learning in the
presence of both selection bias and neighborhood effect, while the previous
debiasing estimators cannot result in unbiased learning without imposing extra
strong assumptions. Extensive semi-synthetic and real-world experiments are
conducted to demonstrate the effectiveness of the proposed methods for
eliminating the selection bias under interference.

##  2 Preliminaries: Previous Selection Bias Formulation

Let $u\in\mathcal{U}$ and $i\in\mathcal{I}$ be a user and an item, $x_{u,i}$,
$o_{u,i}$, and $r_{u,i}$ be the feature, treatment (e.g., exposure), and
feedback (e.g., conversion) of the user-item pair $(u,i)$, where $o_{u,i}$
equals 1 or 0 represents whether the item $i$ is exposed to user $u$ or not.
Let $\mathcal{D}=\\{(u,i)|u\in\mathcal{U},i\in\mathcal{I}\\}$ be the set of
all user-item pairs. Using the potential outcome framework (Rubin, 1974;
Neyman, 1990), let $r_{u,i}(1)$ be the potential feedback that would be
observed if item $i$ had been exposed to user $u$ (i.e., $o_{u,i}$ had been
set to 1). The potential feedback $r_{u,i}(1)$ is observed only when
$o_{u,i}=1$, otherwise it is missing. Then ignoring the missing $r_{u,i}(1)$
and training the prediction model directly with the exposed data suffers from
selection bias, since the exposure is not random and is affected by various
factors.

In the absence of neighborhood effect, the potential feedback $r_{u,i}(1)$
represents the user’s preference by making intervention $o_{u,i}=1$. To
predict $r_{u,i}(1)$ for all $(u,i)\in\mathcal{D}$, let
$\hat{r}_{u,i}\triangleq f_{\theta}(x_{u,i})$ be a prediction model
parameterized with $\theta$. Denote $\hat{\bf
R}\in\mathbb{R}^{|\mathcal{U}|\times|\mathcal{I}|}$ as the predicted potential
feedback matrix with each element being $\hat{r}_{u,i}$. If all the potential
feedback $\\{r_{u,i}(1)|(u,i)\in\mathcal{D}\\}$ were observed, the ideal loss
for training the prediction model $\hat{r}_{u,i}$ is formally defined as

$\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})=|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}\delta(\hat{r}_{u,i},r_{u,i}(1)),$

where $\delta(\cdot,\cdot)$ is a pre-defined loss function, e.g., the squared
loss $(r_{u,i}(1)-\hat{r}_{u,i})^{2}$. However, since $r_{u,i}(1)$ is missing
when $o_{u,i}=0$, the ideal loss cannot be computed directly from
observational data. To tackle this problem, many debiasing methods are
developed to address the selection bias by establishing unbiased estimators of
$\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$, such as error imputation
based (EIB) method (Hernández-Lobato et al., 2014), inverse propensity scoring
(IPS) method (Schnabel et al., 2016), self-normalized IPS (SNIPS) method
(Swaminathan and Joachims, 2015), and doubly robust (DR) methods (Wang et al.,
2019; Chen et al., 2021a; Dai et al., 2022; Li et al., 2023e). We summarize
the causal parameter of interest and the corresponding estimation methods in
the previous studies as follows.

  * •

For the causal parameter of interest, previous studies assume the targeted
user preference $r_{u,i}(o_{u,i}=1)$ depends only on the treatment status
$o_{u,i}=1$. Then the ideal loss is defined using the sample average of
$\delta(\hat{r}_{u,i},r_{u,i}(o_{u,i}=1))$.

  * •

For the methods of estimating the causal parameter of interest, previous
studies have made extensive efforts to estimate the probability
$\mathbb{P}(o_{u,i}=1\mid x_{u,i})$, called propensity, i.e., the probability
of item $i$ exposed to user $u$ given the features $x_{u,i}$. Then the
existing IPS and DR methods use the inverse of the propensity for weighting
the observed samples.

Nevertheless, we argue that both the causal parameter and the corresponding
estimation methods in the previous studies lead to the failure when
eliminating the selection bias under interference.

  * •

(Section 3) For the causal parameter of interest, as shown in Figure 1(b), in
the presence of interference, both the treatment status $o_{u,i}$ and the
treatment statuses $\bm{o}_{\mathcal{N}_{(u,i)}}$ would affect the targeted
user preference $r_{u,i}(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}})$, instead of
$r_{u,i}(o_{u,i})$ in the previous studies.

  * •

(Section 4) For the estimation methods of the causal parameter of interest, as
shown in Figure 1(b), when performing propensity-based reweighting methods,
both $o_{u,i}$ and $\bm{o}_{\mathcal{N}_{(u,i)}}$ from its neighbors should be
considered as treatments of user-item pair $(u,i)$. Therefore, the propensity
should be modeled as $\mathbb{P}(o_{u,i}=1,\bm{o}_{\mathcal{N}_{(u,i)}}\mid
x_{u,i})$ instead of $\mathbb{P}(o_{u,i}=1\mid x_{u,i})$ in previous studies,
which motivates us to design new IPS and DR estimators under interference.

##  3 Modeling Selection Bias under Neighborhood Effect

In this section, we take the neighborhood effect in RS as an interference
problem in causal inference area, and then introduce a treatment
representation to capture the neighborhood effect. Lastly, we propose a novel
ideal loss when both selection bias and neighborhood effect are present.

###  3.1 Beyond “No Interference” Assumption in Previous Studies

In the presence of neighborhood effect, the value of $r_{u,i}(1)$ depends on
not only the user’s preference but also the neighborhood effect, therefore we
cannot distinguish the influence of user preference and the neighborhood
effect, even if all the potential outcomes
$\\{r_{u,i}(1):(u,i)\in\mathcal{D}\\}$ are known. Conceptually, the
neighborhood effect will cause the value of $r_{u,i}(1)$ relying on the
exposure status $o_{u^{\prime},i^{\prime}}$ and the feedback
$r_{u^{\prime},i^{\prime}}$ for some other user-item pairs
$(u^{\prime},i^{\prime})\neq(u,i)$. Formally, we say that interference exists
when a treatment on one unit has an effect on the outcome of another unit
(Ogburn and VanderWeele, 2014; Forastiere et al., 2021; Sävje et al., 2021),
due to the social or physical interaction among units. Previous debiasing
methods rely on the “no interference” assumption, which requires the potential
outcomes of a unit are not affected by the treatment status of the other
units. Nevertheless, such an assumption can hardly be satisfied in real-world
recommendation scenarios.

###  3.2 Proposed Causal Parameter of Interest under Interference

Let $\bm{o}=(o_{1,1},...,o_{|\mathcal{U}|,|\mathcal{I}|})$ be the vector of
exposures of all user-item pairs. For each $(u,i)\in\mathcal{D}$, we define a
partition of
$\bm{o}=(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}},\bm{o}_{\mathcal{N}_{-(u,i)}})$,
where $\mathcal{N}_{(u,i)}$ is all the user-item pairs affecting $(u,i)$,
called the _neighbors_ of $(u,i)$, and $\mathcal{N}_{-(u,i)}$ is all the user-
item pairs not affecting $(u,i)$. When the feedback $r_{u,i}$ is further
influenced by the neighborhood exposures $\bm{o}_{\mathcal{N}_{(u,i)}}$, then
the potential feedback of $(u,i)$ should be defined as
$r_{u,i}(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}})$ to account for the
neighbourhood effect.

However, if we take $(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}})$ as the new
treatment directly, it would be a high-dimensional sparse vector when the
dimension of $\bm{o}_{\mathcal{N}_{(u,i)}}$ is high and the number of exposed
neighbors is limited. To address this problem and capture the neighborhood
effect effectively, we make an assumption on the interference mechanism
leveraging the idea of representation learning (Johansson et al., 2016).

######  Assumption 1 (Neighborhood Treatment Representation).

There exists a representation vector
$\bm{\phi}:\\{0,1\\}^{|\mathcal{N}_{(u,i)}|}\to\mathcal{G}$, if
$\bm{\phi}(\bm{o}_{\mathcal{N}_{(u,i)}})=\bm{\phi}(\bm{o}^{\prime}_{\mathcal{N}_{(u,i)}})$,
then
$r_{u,i}(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}})=r_{u,i}(o_{u,i},\bm{o}^{\prime}_{\mathcal{N}_{(u,i)}})$.

The above assumption implies that the value of
$r_{u,i}(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}})$ depends on
$\bm{o}_{\mathcal{N}_{(u,i)}}$ through a specific treatment representation
$\bm{\phi}(\cdot)$ that summarizes the neighborhood effect. Denote
$\bm{g}_{u,i}=\bm{\phi}(\bm{o}_{\mathcal{N}_{(u,i)}})$, then we have
$r_{u,i}(o_{u,i},\bm{o}_{\mathcal{N}_{(u,i)}})=r_{u,i}(o_{u,i},\bm{g}_{u,i})$
under Assumption 1, i.e., the feedback of $(u,i)$ under individual exposure
$o_{u,i}$ and treatment representation $\bm{g}_{u,i}$.

We now propose ideal loss under neighborhood effect with treatment
representation level $\bm{g}\in\mathcal{G}$ as

$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})=|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}\delta(\hat{r}_{u,i},r_{u,i}(o_{u,i}=1,\bm{g})),$

and the final ideal loss summarizes various neighborhood effects
$\bm{g}\in\mathcal{G}$ as

$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})=\int\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g},$

where $\pi(\bm{g})$ is a pre-specified probability density function of
$\bm{g}$.

The proposed $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ forces
the prediction model $\hat{r}_{u,i}$ to perform well across varying treatment
representation levels $\bm{g}\in\mathcal{G}$. Thus,
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ is expected to
control the extra bias that arises from the neighborhood effect. In
comparison, the self interest and neighborhood effect are intertwined in
previously used $\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$, whereas our
proposed $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ is very
flexible due to the free choice of $\pi(\bm{g})$. The choice of $\pi(\bm{g})$
depends on the target population that we want to make predictions on. Consider
an extreme case of no neighborhood effect, this corresponds to
$\bm{g}_{u,i}=0$ for all user-item pairs. In such a case, we can write
$r_{u,i}(1,0)$ as $r_{u,i}(1)$ and
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ would reduce to
$\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$.

##  4 Unbiased Estimation and Learning under Interference

In this section, we first discuss the consequence of ignoring the neighborhood
effect, and then propose two novel estimators for estimating the ideal loss
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$. Moreover, we
theoretically analyze the bias, variance, optimal bandwidth, and
generalization error bounds of the proposed estimators.

Before presenting the proposed debiasing methods under interference, we
briefly discuss the identifiability of the ideal loss
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$. A causal estimand is
said to be identifiable if it can be written as a series of quantities that
can be estimated from observed data.

######  Assumption 2 (Consistency under Interference).

$r_{u,i}=r_{u,i}(1,\bm{g})$ if $o_{u,i}=1$ and $\bm{g}_{u,i}=\bm{g}$.

######  Assumption 3 (Unconfoundedness under Interference).

$r_{u,i}(1,\bm{g})\perp\\!\\!\\!\perp(o_{u,i},\bm{G}_{u,i})\mid x_{u,i}$.

These assumptions are common in causal inference to ensure the identifiability
of causal effects. Specifically, Assumption 2 implies that $r_{u,i}(1,\bm{g})$
is observed only when $o_{u,i}=1$ and $\bm{g}_{u,i}=\bm{g}$. Assumption 3
indicates that there is no unmeasured confounder that affects both $r_{u,i}$
and $(o_{u,i},\bm{g}_{u,i})$.

######  Theorem 1 (Identifiability).

Under Assumptions 1–3, $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})$ and $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ are
identifiable.

Theorem 1 ensures the identifiability of the proposed ideal loss
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$. Let $\mathbb{E}$
denote the expectation on the target population $\mathcal{D}$, and $p(\cdot)$
denotes the probability density function of $\mathbb{P}$.

###  4.1 Effect of Ignoring Interference

The widely used ideal loss $\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$
under no neighborhood effect is generally different from the proposed ideal
loss $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ in the presence
of neighborhood effect. Next, we establish the connection between these two
loss functions, to deepen the understanding of the methods of
considering/ignoring neighborhood effect. For brevity, we let
$\delta_{u,i}(\bm{g})=\delta(\hat{r}_{u,i},r_{u,i}(1,\bm{g}))$ hereafter.

######  Theorem 2 (Link to Selection Bias).

Under Assumptions 1–3,

(a) if $\bm{g}_{u,i}\perp\\!\\!\\!\perp o_{u,i}\mid x_{u,i}$,
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})=\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$;

(b) if $\bm{g}_{u,i}\not\\!\perp\\!\\!\\!\perp o_{u,i}\mid x_{u,i}$,
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})-\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$ is equal to

$\displaystyle\int\mathbb{E}\Big{[}\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\cdot\Big{\\{}p(\bm{g}_{u,i}=\bm{g}|x_{u,i})-p(\bm{g}_{u,i}=\bm{g}|x_{u,i},o_{u,i}=1)\Big{\\}}\Big{]}\pi(\bm{g})d\bm{g}.$

From Theorem 2(a), if the individual and neighborhood exposures are
independent conditional on $x_{u,i}$, then
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ is equal to
$\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$, which indicates that the
existing debiasing methods neglecting neighborhood effect are also unbiased
estimator of $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$. This is
intuitively reasonable since in such a case, the neighborhood effect randomly
influences $o_{u,i}$ conditional on $x_{u,i}$, and the effect of neighbors
would be smoothed out in an average sense. Theorem 2(b) shows that a bias
would arise when $\bm{g}_{u,i}\not\\!\perp\\!\\!\\!\perp o_{u,i}\mid x_{u,i}$,
and the bias mainly depends on the association between $o_{u,i}$ and
$\bm{g}_{u,i}$ conditional on $x_{u,i}$, i.e.,
$p(\bm{g}_{u,i}=\bm{g}|x_{u,i}=x)-p(\bm{g}_{u,i}=\bm{g}|x_{u,i}=x,o_{u,i}=1)$.

###  4.2 Proposed Unbiased Estimators

To derive an unbiased estimator of
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$, it suffices to find
an unbiased estimator of $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})$. Motivated by Schnabel et al. (2016), an intuitive solution is to
take $(o_{u,i},\bm{g}_{u,i})$ as a joint treatment, then the IPS estimator of
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R}|\bm{g})$ should be
$|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}\mathbb{I}\\{o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\\}\cdot\delta_{u,i}(\bm{g})/p_{u,i}(\bm{g}),$
where $\mathbb{I}(\cdot)$ is an indicator function,
$p_{u,i}(\bm{g})=p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})$ is the propensity
score. Clearly, this strategy works if $\bm{g}_{u,i}$ is a binary or multi-
valued random variable. However, if $\bm{g}_{u,i}$ has a continuous
probability density, the above estimator is numerically infeasible even if
theoretically feasible, since almost all
$\mathbb{I}\\{o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\\}$ will be zero in such a case.

To tackle this problem, we propose a novel kernel-smoothing based neighborhood
IPS (N-IPS) estimator of $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})$, which is given as

$\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})=|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot
p_{u,i}(\bm{g})},$

where $h$ is a bandwidth (smoothing parameter) and $K(\cdot)$ is a symmetric
kernel function (Fan and Gijbels, 1996; Li and Racine, 2023; Wu et al., 2024)
that satisfies $\int K(t)dt=1$ and $\int tK(t)dt=1$. For example, Epanechnikov
kernel $K(t)=3(1-t^{2})\mathbb{I}\\{|t|\leq 1\\}/4$ and Gaussian kernel
$K(t)=\exp(-t^{2}/2)/\sqrt{2\pi}$ for $t\in\mathbb{R}$. For ease of
presentation, we state the results for a scalar $\bm{g}$. Similar conclusions
can be derived for multi-dimensional $\bm{g}$ and we put them in Appendix C.

Similarly, the kernel-smoothing based neighborhood DR (N-DR) estimator can be
constructed by

$\displaystyle\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})=|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}\Big{[}\hat{\delta}_{u,i}(\bm{g})+\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}}{h\cdot
p_{u,i}(\bm{g})}\Big{]},$

where
$\hat{\delta}_{u,i}(\bm{g})=\delta(\hat{r}_{u,i},m(x_{u,i},\phi_{\bm{g}}))$ is
the imputed error of $\delta_{u,i}(\bm{g})$, and $m(x_{u,i},\phi_{\bm{g}})$ is
an imputation model of $r_{u,i}(1,\bm{g})$. The imputation model is trained by
minimizing the training loss

$\mathcal{L}_{e}^{\mathrm{N-DR}}(\hat{\bf
R})=\int|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot(\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g}))^{2}}{h\cdot
p_{u,i}(\bm{g})}}\pi(\bm{g})d\bm{g}.$

Then, the corresponding N-IPS and N-DR estimators of
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ are given as

$\displaystyle\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})=\int\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g},\quad\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})=\int\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g}.$ (1)

Next, we show the bias and variance of the proposed N-IPS and N-DR estimators,
which rely on a standard assumption in kernel-smoothing estimation (Härdle et
al., 2004; Li and Racine, 2023).

######  Assumption 4 (Regularity Conditions for Kernel Smoothing).

(a) $h\to 0$ as $|\mathcal{D}|\to\infty$; (b) $|\mathcal{D}|h\to\infty$ as
$|\mathcal{D}|\to\infty$; (c) $p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\mid x_{u,i})$
is twice differentiable with respect to $\bm{g}$.

######  Theorem 3 (Bias and Variance of N-IPS and N-DR).

Under Assumptions 1–4,

(a) the bias of the N-DR estimator is given as

$\text{Bias}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2}),$

where $\mu_{2}=\int K(t)t^{2}dt$. The bias of N-IPS is provided in Appendix
B.2;

(b) the variance of the N-DR estimator is given as

$\text{Var}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{|\mathcal{D}|h}\int\psi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}),$

where
$\psi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\\{\delta_{u,i}(\bm{g}^{\prime})-\hat{\delta}_{u,i}(\bm{g}^{\prime})\\}\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$
is a bounded function of $\bm{g}$, $\bar{K}(\cdot)=\int
K\left(t\right)K\left(\cdot+t\right)dt$. The variance of N-IPS is provided in
Appendix B.2.

From Theorem 3(a), the kernel-smoothing based N-DR estimator has a small bias
of order $O(h^{2})$, which converges to 0 as $|\mathcal{D}|\to\infty$ by
Assumption 4(a). Theorem 3(b) shows that the variance of the N-DR estimator
has a convergence rate of order $O(1/|\mathcal{D}|h)$. Notably, the bandwidth
$h$ plays a key role in the bias-variance trade-off of the N-DR estimator: the
larger the $h$, the larger the bias and the smaller the variance. The
following Theorem 4 gives the optimal bandwidth for N-IPS and N-DR.

######  Theorem 4 (Optimal Bandwidth of N-IPS and N-DR).

Under Assumptions 1–4, the optimal bandwidth for the N-DR estimator in terms
of the asymptotic mean-squared error is

$h^{*}_{\mathrm{N-DR}}=\left[\frac{\int\psi(\bm{g})\pi(\bm{g})d\bm{g}}{4|\mathcal{D}|\left(\frac{1}{2}\mu_{2}\int\mathbb{E}\left[\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\right]\pi(\bm{g})d\bm{g}\right)^{2}}\right]^{1/5},$

where $\psi(\bm{g})$ is defined in Theorem 3. The optimal bandwidth for N-IPS
is provided in Appendix B.3.

Theorem 4 shows that the optimal bandwidth of N-DR is of order
$O(|\mathcal{D}|^{-1/5})$. In such a case,

$\left[\text{Bias}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))\right]^{2}=O(h^{4})=O(|\mathcal{D}|^{-4/5}),\quad\text{Var}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=O(\frac{1}{|\mathcal{D}|h})=O(|\mathcal{D}|^{-4/5}),$

that is, the square of the bias has the same convergence rate as the variance.

###  4.3 Propensity Estimation Method

Different from previous debiasing methods in RS, in the presence of
neighborhood effect, the propensity is defined for joint treatment that
includes a binary variable $o$ and a continuous variable $\bm{g}$. To fill
this gap, we consider a novel method for propensity estimation. Let
$\mathbb{P}^{u}(\bm{g}\mid o=1,\bm{x})$ be a uniform distribution on
$\mathcal{G}$ and equals ${1}/{c}$ for all feature $\bm{x}$. Note that

$\frac{1}{p_{u,i}(\bm{g})}=\frac{1}{\mathbb{P}(o=1\mid\bm{x})\mathbb{P}(\bm{g}\mid
o=1,\bm{x})}=\frac{c}{\mathbb{P}(o=1\mid\bm{x})}\cdot\frac{\mathbb{P}^{u}(\bm{g}\mid
o=1,\bm{x})}{\mathbb{P}(\bm{g}\mid o=1,\bm{x})},$

where $\mathbb{P}(o=1\mid\bm{x})$ can be estimated by using the existing
methods such as naive Bayes or logistic regression with or without a few
unbiased ratings, respectively (Schnabel et al., 2016). To estimate the
density ratio $\mathbb{P}^{u}(\bm{g}\mid o=1,\bm{x})/\mathbb{P}(\bm{g}\mid
o=1,\bm{x})$, we first label the samples in the exposed data
$\left\\{\left(\bm{x}_{u,i},\bm{g}_{u,i}\right)\right\\}_{\\{(u,i):o_{u,i}=1\\}}$
as positive samples $(L=1)$, then uniformly sample treatments
$\bm{g}_{u,i}^{\prime}\in\mathcal{G}$ to generate samples
$\left\\{(\bm{x}_{u,i},\bm{g}_{u,i}^{\prime})\right\\}_{\\{(u,i):o_{u,i}=1\\}}$
with negative labels $(L=0)$. Since the data generating process ensures that
$\mathbb{P}^{u}(\bm{x}\mid o=1)=\mathbb{P}(\bm{x}\mid o=1)$, we have

$\frac{\mathbb{P}^{u}(\bm{g}\mid o=1,\bm{x})}{\mathbb{P}(\bm{g}\mid
o=1,\bm{x})}=\frac{\mathbb{P}^{u}(\bm{x},\bm{g}\mid
o=1)}{\mathbb{P}(\bm{x},\bm{g}\mid o=1)}=\frac{\mathbb{P}(\bm{x},\bm{g}\mid
L=0)}{\mathbb{P}(\bm{x},\bm{g}\mid
L=1)}=\frac{\mathbb{P}(L=1)}{\mathbb{P}(L=0)}\cdot\frac{\mathbb{P}(L=0\mid\bm{x},\bm{g})}{\mathbb{P}(L=1\mid\bm{x},\bm{g})},$

where $\mathbb{P}(L=l\mid\bm{x},\bm{g})$ for $l=0$ or $1$ can be obtained by
modeling $L$ with $(\bm{x},\bm{g})$.

###  4.4 Further Theoretical Analysis

We further theoretically analyze the generalization error bounds of the
proposed N-IPS and N-DR estimators. Letting $\mathcal{F}$ be the hypothesis
space of prediction matrices $\hat{\bf R}$ (or prediction model $f_{\theta}$),
we define the Rademacher complexity

$\mathcal{R}(\mathcal{F})=\mathbb{E}_{\bm{\sigma}\sim\\{-1,+1\\}^{|\mathcal{D}|}}\sup_{f_{\theta}\in\mathcal{F}}\Big{[}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\sigma_{u,i}\delta_{u,i}(\bm{g})\Big{]},$

where $\bm{\sigma}=\\{\sigma_{u,i}:(u,i)\in\mathcal{D}\\}$ is a Rademacher
sequence (Mohri et al., 2018).

######  Assumption 5 (Boundedness).

$1/p_{u,i}(\bm{g})\leq M_{p}$, $\delta_{u,i}(\bm{g})\leq M_{\delta}$, and
$|\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})|\leq
M_{|\delta-\hat{\delta}|}$.

Theorem 5 gives the generalization error bounds of the prediction model
trained by minimizing our proposed N-IPS and N-DR estimators.

######  Theorem 5 (Generalization Error Bounds of N-IPS and N-DR).

Under Assumptions 1–5 and suppose that $K(t)\leq M_{K}$, we have with
probability at least $1-\eta$,

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})\leq{}$ $\displaystyle\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})+\mu_{2}M_{|\delta-\hat{\delta}|}\Big{|}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\Big{|}\cdot
h^{2}$
$\displaystyle+\frac{4M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{K}M_{|\delta-\hat{\delta}|}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}+o(h^{2}),$

where $\hat{\bf R}^{\dagger}=\arg\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})$ is the
learned prediction model by minimizing the N-DR estimator. The generalization
error bounds of the N-IPS estimator is provided in Appendix B.4.

##  5 Semi-synthetic Experiments

We conduct semi-synthetic experiments using the MovieLens
100K111https://grouplens.org/datasets/movielens/100k/ (ML-100K) dataset,
focusing on the following two research questions (RQs): RQ1. Do the proposed
estimators result in more accurate estimation for ideal loss compared to the
previous estimators in the presence of neighborhood effect? RQ2. How does the
neighborhood effect strength affect the estimation accuracy?

Experimental Setup222Our codes and datasets are available at
https://github.com/haoxuanli-pku/ICLR24-Interference.. The ML-100K dataset
contains 100,000 missing-not-at-random (MNAR) ratings from 943 users to 1,682
movies. Following the previous studies (Schnabel et al., 2016; Wang et al.,
2019; Guo et al., 2021), we first complete the full rating matrix $\mathbf{R}$
by Matrix Factorization (MF) (Koren et al., 2009), resulting in
$r_{u,i}\in\\{1,2,3,4,5\\}$, and then set propensity
$p_{u,i}=p\alpha^{\max\left(0,4-r_{u,i}\right)}$ with $\alpha=0.5$ to model
MNAR effect (Wang et al., 2019; Guo et al., 2021). Next, to model the
neighborhood effect, we compute
$g_{u,i}=\mathbb{I}(\sum_{(u^{\prime},i^{\prime})\in\mathcal{N}_{(u,i)}}o_{u^{\prime},i^{\prime}}\geq
c)$ with varying $c$ for 100,000 observed MNAR ratings, where
$\mathcal{N}_{(u,i)}=\\{({u^{\prime}},i^{\prime})\neq(u,i)\mid
u^{\prime}=u~{}\text{or}~{}i^{\prime}=i\\}$. In our experiment, $c$ is chosen
to be the median of all
$\sum_{(u^{\prime},i^{\prime})\in\mathcal{N}_{(u,i)}}o_{u^{\prime},i^{\prime}}$
for $(u,i)\in\mathcal{D}$. Then we complete two full rating matrices
$\mathbf{R}^{g=0}$ and $\mathbf{R}^{g=1}$ with
$r_{u,i}(1,g)\in\\{1,2,3,4,5\\}$ by MF, using $\\{(u,i)\mid
o_{u,i}=1,g_{u,i}=0\\}$ and $\\{(u,i)\mid o_{u,i}=1,g_{u,i}=1\\}$
respectively.

Experimental Details. The computation of the ideal loss needs both a ground-
truth rating matrix and a predicted rating matrix. Therefore, we generate the
following six predicted matrices $\mathbf{\hat{R}}$:

$\bullet$ ONE: The predicted rating matrix $\hat{\mathbf{R}}$ is identical to
the true rating matrix, except that $|\\{(u,i)\mid r_{u,i}=5\\}|$ randomly
selected true ratings of 1 are flipped to 5. This means half of the predicted
fives are true five, and half are true one.
$\bullet$ THREE: Same as ONE, but flipping true rating of 3.
$\bullet$ FOUR: Same as ONE, but flipping true rating of 4.
$\bullet$ ROTATE: For each predicted rating $\hat{r}_{u,i}=r_{u,i}-1$ when
$r_{u,i}\geq 2$, and $\hat{r}_{u,i}=5$ when $r_{u,i}=1$.
$\bullet$ SKEW: Predicted $\hat{r}_{u,i}$ are sampled from the Gaussian
distribution $\mathcal{N}(\mu=r_{u,i},\sigma=(6-r_{u,i})/{2})$, and clipped to
the interval $[1,5]$.
$\bullet$ CRS: Set $\hat{r}_{u,i}=2$ if $r_{u,i}\leq 3$, otherwise, set
$\hat{r}_{u,i}=4$.

To consider the neighborhood effect, we assume that each user-item pair in the
uniform data has an equal probability of having $g_{u,i}=0$ and $g_{u,i}=1$,
that is, $\pi(g)=0.5$ for $g\in\\{0,1\\}$. Thus,
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})=|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}\\{\delta(\hat{r}_{u,i},r_{u,i}(1,g=0))+\delta(\hat{r}_{u,i},r_{u,i}(1,g=1)\\}/2$,
where $\delta(\cdot,\cdot)$ is the mean absolute error (MAE). We follow the
previous studies (Guo et al., 2021; Li et al., 2023b) to adopt relative
absolute error (RE) to measure the estimation accuracy, which is defined as
$\operatorname{RE}(\mathcal{L}_{\mathrm{est}})=|\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})-\mathcal{L}_{{\mathrm{est}}}(\hat{\mathbf{R}})|/\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})$, where $\mathcal{L}_{\mathrm{est}}$ denotes the ideal loss estimation by
the estimator. The smaller the RE, the higher the estimation accuracy (see
Appendix E for more details).

Table 1: Relative error on six prediction metrics. The best results are
bolded.

| ONE | THREE | FOUR | ROTATE | SKEW | CRS
---|---|---|---|---|---|---
Naive | 0.8612 ± 0.0068 | 1.0011 ± 0.0075 | 1.0471 ± 0.0077 | 0.2781 ± 0.0019 | 0.3538 ± 0.0038 | 0.3419 ± 0.0030
IPS | 0.4766 ± 0.0060 | 0.5501 ± 0.0056 | 0.5731 ± 0.0057 | 0.1434 ± 0.0040 | 0.1969 ± 0.0046 | 0.1885 ± 0.0028
N-IPS | 0.2383 ± 0.0066 | 0.2670 ± 0.0069 | 0.2829 ± 0.0062 | 0.0417 ± 0.0043 | 0.1024 ± 0.0051 | 0.0966 ± 0.0029
DR | 0.4247 ± 0.0088 | 0.4637 ± 0.0093 | 0.4661 ± 0.0096 | 0.0571 ± 0.0021 | 0.1938 ± 0.0043 | 0.0565 ± 0.0020
N-DR | 0.3089 ± 0.0088 | 0.3533 ± 0.0091 | 0.3577 ± 0.0092 | 0.0339 ± 0.0031 | 0.1219 ± 0.0039 | 0.0511 ± 0.0026
MRDR | 0.2578 ± 0.0070 | 0.2639 ± 0.0071 | 0.2611 ± 0.0073 | 0.1001 ± 0.0025 | 0.1538 ± 0.0038 | 0.0156 ± 0.0021
N-MRDR | 0.0622 ± 0.0065 | 0.0520 ± 0.0065 | 0.0503 ± 0.0064 | 0.0456 ± 0.0037 | 0.0672 ± 0.0038 | 0.0042 ± 0.0022

(a) REC ONE

(b) REC THREE

(c) REC FOUR

(d) ROTATE

(e) SKEW

(f) CRS

Figure 2: The effect of mask numbers as interference strength on RE on six
prediction matrices. Table 2: Performance of MSE, AUC, and NDCG@5 on three
real-world datasets. The best six results are bolded, and the best baseline is
underlined.

Dataset | Coat | Yahoo! R3 | KuaiRec
---|---|---|---
Method | MSE $\downarrow$ | AUC $\uparrow$ | N@5 $\uparrow$ | MSE $\downarrow$ | AUC $\uparrow$ | N@5 $\uparrow$ | MSE $\downarrow$ | AUC $\uparrow$ | N@50 $\uparrow$
Base model (Koren et al., 2009) | 0.238 | 0.710 | 0.616 | 0.249 | 0.682 | 0.634 | 0.137 | 0.754 | 0.553
\+ CVIB (Wang et al., 2020) | 0.222 | 0.722 | 0.635 | 0.257 | 0.683 | 0.645 | 0.103 | 0.769 | 0.563
\+ DIB (Liu et al., 2021) | 0.242 | 0.726 | 0.629 | 0.248 | 0.687 | 0.641 | 0.142 | 0.754 | 0.556
\+ SNIPS (Schnabel et al., 2016) | 0.208 | 0.737 | 0.636 | 0.245 | 0.687 | 0.656 | 0.048 | 0.788 | 0.576
\+ ASIPS (Saito, 2020a) | 0.205 | 0.722 | 0.621 | 0.230 | 0.678 | 0.643 | 0.097 | 0.753 | 0.554
\+ DAMF (Saito and Nomura, 2022) | 0.218 | 0.734 | 0.643 | 0.245 | 0.697 | 0.656 | 0.097 | 0.775 | 0.572
\+ DR (Saito, 2020b) | 0.208 | 0.726 | 0.634 | 0.216 | 0.684 | 0.658 | 0.046 | 0.773 | 0.564
\+ DR-BIAS (Dai et al., 2022) | 0.223 | 0.717 | 0.631 | 0.220 | 0.689 | 0.654 | 0.046 | 0.771 | 0.552
\+ DR-MSE (Dai et al., 2022) | 0.214 | 0.720 | 0.630 | 0.222 | 0.689 | 0.657 | 0.047 | 0.769 | 0.547
\+ MR (Li et al., 2023a) | 0.210 | 0.730 | 0.643 | 0.247 | 0.693 | 0.651 | 0.114 | 0.780 | 0.573
\+ TDR (Li et al., 2023b) | 0.229 | 0.710 | 0.634 | 0.234 | 0.674 | 0.662 | 0.134 | 0.769 | 0.573
\+ TDR-JL (Li et al., 2023b) | 0.216 | 0.734 | 0.639 | 0.248 | 0.684 | 0.654 | 0.121 | 0.771 | 0.560
\+ SDR (Li et al., 2023e) | 0.208 | 0.736 | 0.642 | 0.210 | 0.690 | 0.655 | 0.116 | 0.775 | 0.574
\+ IPS (Schnabel et al., 2016) | 0.214 | 0.718 | 0.626 | 0.221 | 0.681 | 0.644 | 0.097 | 0.752 | 0.554
\+ N-IPS [LR, Gaussian] | 0.212 | 0.742 | 0.678 | 0.226 | 0.693 | 0.664 | 0.092 | 0.796 | 0.585
\+ N-IPS [LR, Epanechnikov] | 0.224 | 0.746 | 0.645 | 0.242 | 0.703 | 0.673 | 0.094 | 0.794 | 0.582
\+ N-IPS [NB, Gaussian] | 0.206 | 0.744 | 0.648 | 0.196 | 0.693 | 0.658 | 0.049 | 0.785 | 0.579
\+ N-IPS [NB, Epanechnikov] | 0.210 | 0.753 | 0.646 | 0.197 | 0.685 | 0.653 | 0.047 | 0.755 | 0.562
\+ DR-JL (Wang et al., 2019) | 0.211 | 0.721 | 0.620 | 0.224 | 0.682 | 0.646 | 0.050 | 0.764 | 0.526
\+ N-DR-JL [LR, Gaussian] | 0.231 | 0.731 | 0.651 | 0.247 | 0.698 | 0.664 | 0.113 | 0.779 | 0.537
\+ N-DR-JL [LR, Epanechnikov] | 0.235 | 0.741 | 0.655 | 0.251 | 0.693 | 0.663 | 0.108 | 0.784 | 0.552
\+ N-DR-JL [NB, Gaussian] | 0.204 | 0.748 | 0.650 | 0.198 | 0.691 | 0.653 | 0.049 | 0.778 | 0.574
\+ N-DR-JL [NB, Epanechnikov] | 0.209 | 0.744 | 0.648 | 0.191 | 0.681 | 0.637 | 0.046 | 0.786 | 0.570
\+ MRDR-JL (Guo et al., 2021) | 0.214 | 0.721 | 0.631 | 0.215 | 0.686 | 0.650 | 0.047 | 0.777 | 0.554
\+ N-MRDR-JL [LR, Gaussian] | 0.217 | 0.728 | 0.662 | 0.252 | 0.697 | 0.666 | 0.107 | 0.785 | 0.539
\+ N-MRDR-JL [LR, Epanechnikov] | 0.233 | 0.734 | 0.656 | 0.253 | 0.695 | 0.666 | 0.097 | 0.791 | 0.560
\+ N-MRDR-JL [NB, Gaussian] | 0.208 | 0.742 | 0.651 | 0.206 | 0.694 | 0.663 | 0.045 | 0.793 | 0.583
\+ N-MRDR-JL [NB, Epanechnikov] | 0.207 | 0.756 | 0.635 | 0.194 | 0.690 | 0.644 | 0.044 | 0.802 | 0.587

Performance Analysis. We take three propensity-based estimators: IPS, DR, and
MRDR as baselines (see Section 6 for baselines introduction). The results are
shown in Table 5. First, the REs of our estimators are significantly lower
compared to the corresponding previous estimators, which indicates that our
estimators are able to estimate the ideal loss accurately in the presence of
neighborhood effect. In addition, to investigate how the neighborhood effect
affects the estimation error, we randomly mask some user rows and item columns
before sampling $o_{u,i}$, which results in $p_{u,i}=0$ for the masked user-
item pairs. For unmasked user-item pairs, we raise their propensities such
that the expected total number of observed samples remains the same, which
increases the proportion of observed samples with $g_{u,i}=1$ to strengthen
the neighborhood effect. Figure 2 shows the RE of the estimators with varying
neighborhood effects. Our methods stably outperform the previous methods in
all scenarios, which verifies that our methods are robust to the increased
neighborhood effect.

##  6 Real-World Experiments

Dataset and Experiment Details. We verify the effectiveness of the proposed
estimators on three real-world datasets: Coat contains 6,960 MNAR ratings and
4,640 missing-at-random (MAR) ratings. Yahoo! R3 contains 311,704 MNAR ratings
and 54,000 MAR ratings. KuaiRec contains 4,676,570 video watching ratio
records from 1,411 users for 3,327 videos. We pre-specify three neighborhood
choices for a user-item pair in MNAR data: (1) using user historical behavior,
(2) using the purchase history of an item, and (3) using the interaction of
users and items, and let $g_{u,i}$ be the neighborhood number of the user-item
pair, which is a multi-valued representation. We report the best result of our
methods among the three neighborhood choices using MSE, AUC, and NDCG@$K$ as
the evaluation protocols, where $K=5$ for Coat and Yahoo! R3 and $K=50$ for
KuaiRec. We adopt both the Gaussian kernel and Epanechnikov kernel as the
kernel function for implementing our proposed N-IPS, N-DR-JL, and N-MRDR (see
Appendix F for more details).

Baselines. We take Matrix Factorization (MF) (Koren et al., 2009) as the base
model and consider the following debiasing baselines: IPS (Schnabel et al.,
2016; Saito et al., 2020), SNIPS (Schnabel et al., 2016), DR-JL (Wang et al.,
2019), ASIPS (Saito, 2020a), CVIB (Wang et al., 2020), DR (Saito, 2020b),
MRDR-JL (Guo et al., 2021), DIB (Liu et al., 2021), DAMF (Saito and Nomura,
2022), DR-BIAS (Dai et al., 2022), DR-MSE (Dai et al., 2022), MR (Li et al.,
2023a), Stable-DR (Li et al., 2023e), TDR (Li et al., 2023b), and TDR-JL (Li
et al., 2023b). Following previous studies (Schnabel et al., 2016; Wang et
al., 2019), for all baseline methods requiring propensity estimation, we adopt
naive Bayes (NB) method using 5% MAR ratings for training the propensity
model. For our proposed methods, we also adopt logistic regression (LR) to
estimate the propensities without MAR ratings.

Real-World Debiasing Performance. Table 5 shows the performance of the
baselines and our methods on three datasets. Compared with the base model, the
debiasing methods achieve better performance. Notably, the proposed methods
can stably outperform the baseline methods in all metrics, showing that our
methods can effectively take the neighbor effect into account. This also
provides empirical evidence of the existence of the neighborhood effect in
real-world datasets. The proposed methods show competitive performance whether
the MAR data are accessible (NB) or not (LR), and perform similarly in the
case of adopting the Gaussian kernel or Epanechnikov kernel.

##  7 Conclusion

In this paper, we study the problem of selection bias in the presence of
neighborhood effect. First, we formulate the neighborhood effect in RS as an
interference problem in causal inference. Next, a neighborhood treatment
representation vector is introduced to reduce the dimension and sparsity of
the neighborhood treatments. Based on it, we reformulate the potential
feedback and propose a novel ideal loss that can be used to deal with
selection bias in the presence of neighborhood effect. Then, we propose two
novel kernel-smoothing based neighborhood estimators for the ideal loss, which
allows the neighborhood treatment representation vector to have continuous
probability density. We systematically analyze the properties of the proposed
estimators, including the bias, variance, optimal bandwidth, and
generalization error bounds. In addition, we also theoretically establish the
connection between the debiasing methods considering and ignoring the
neighborhood effect. Extensive experiments are conducted on semi-synthetic and
real-world data to demonstrate the effectiveness of our approaches. A
limitation of this work is that the hypothesis space $\mathcal{G}$ of $\bm{g}$
relies on prior knowledge, and it is not obvious to choose it in practice. We
leave it for our future work.

##  8 Acknowledgement

This work was supported in part by National Natural Science Foundation of
China (No. 623B2002, 12301370).

## References

  * Aronow and Samii [2017] Peter M. Aronow and Cyrus Samii.  Estimating average causal effects under general interference, with application to a social network experiment.  _The Annals of Applied Statistics_ , 11:1912–1947, 2017.
  * Bai et al. [2024] Shuanghao Bai, Min Zhang, Wanqi Zhou, Siteng Huang, Zhirong Luan, Donglin Wang, and Badong Chen.  Prompt-based distribution alignment for unsupervised domain adaptation.  In _AAAI_ , 2024.
  * Chen et al. [2021a] Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang.  Autodebias: Learning to debias for recommendation.  In _SIGIR_ , 2021a.
  * Chen et al. [2023] Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He.  Bias and debias in recommender system: A survey and future directions.  _ACM Transactions on Information Systems_ , 41(3):1–39, 2023.
  * Chen et al. [2021b] Mouxiang Chen, Chenghao Liu, Jianling Sun, and Steven CH Hoi.  Adapting interactional observation embedding for counterfactual learning to rank.  In _SIGIR_ , 2021b.
  * Dai et al. [2022] Quanyu Dai, Haoxuan Li, Peng Wu, Zhenhua Dong, Xiao-Hua Zhou, Rui Zhang, Xiuqiang He, Rui Zhang, and Jie Sun.  A generalized doubly robust learning framework for debiasing post-click conversion rate prediction.  In _KDD_ , 2022.
  * Ding et al. [2022] Sihao Ding, Peng Wu, Fuli Feng, Xiangnan He, Yitong Wang, Yong Liao, and Yongdong Zhang.  Addressing unmeasured confounder for recommendation with sensitivity analysis.  In _KDD_ , 2022.
  * Fan and Gijbels [1996] Jianqing Fan and Irene Gijbels.  _Local Polynomial Modelling and Its Applications_.  Chapman and Hall/CRC, 1996.
  * Ferracci et al. [2014] Marc Ferracci, Grégory Jolivet, and Gerard J. van den Berg.  Evidence of treatment spillovers within markets.  _Review of Economics and Statistics_ , 96:812–823, 2014.
  * Forastiere et al. [2021] Laura Forastiere, Edoardo M. Airoldi, and Fabrizia Mealli.  Identification and estimation of treatment and interference effects in observational studies on networks.  _Journal of the American Statistical Association_ , 116:901–918, 2021.
  * Gao et al. [2022] Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua.  KuaiRec: A fully-observed dataset and insights for evaluating recommender systems.  In _CIKM_ , 2022.
  * Guo et al. [2021] Siyuan Guo, Lixin Zou, Yiding Liu, Wenwen Ye, Suqi Cheng, Shuaiqiang Wang, Hechang Chen, Dawei Yin, and Yi Chang.  Enhanced doubly robust learning for debiasing post-click conversion rate estimation.  In _SIGIR_ , 2021.
  * Hernández-Lobato et al. [2014] José Miguel Hernández-Lobato, Neil Houlsby, and Zoubin Ghahramani.  Probabilistic matrix factorization with non-random missing data.  In _ICML_ , 2014.
  * Hong and Raudenbush [2006] Guanglei Hong and Stephen W. Raudenbush.  valuating kindergarten retention policy: A case study of causal inference for multilevel observational data.  _Journal of the American Statistical Association_ , 101:901–910, 2006.
  * Huang et al. [2022] Jin Huang, Harrie Oosterhuis, and Maarten de Rijke.  It is different when items are older: Debiasing recommendations when selection bias and user preferences are dynamic.  In _WSDM_ , 2022.
  * Huang et al. [2023] Shanshan Huang, Haoxuan Li, Qingsong Li, Chunyuan Zheng, and Li Liu.  Pareto invariant representation learning for multimedia recommendation.  In _ACM-MM_ , 2023.
  * Hudgens and Halloran [2008] Michael G Hudgens and M Elizabeth Halloran.  Toward causal inference with interference.  _Journal of the American Statistical Association_ , 103:832–842, 2008.
  * Härdle et al. [2004] Wolfgang Härdle, Axel Werwatz, Marlene Müller, and Stefan Sperlich.  _Nonparametric and Semiparametric Models_.  Springer Series in Statistics, 2004.
  * Imbens and Rubin [2015] Guido W. Imbens and Donald B. Rubin.  _Causal Inference For Statistics Social and Biomedical Science_.  Cambridge University Press, 2015.
  * Johansson et al. [2016] Fredrik D. Johansson, Uri Shalit, and David Sontag.  Learning representations for counterfactual inference.  In _ICML_ , 2016.
  * Koren et al. [2009] Yehuda Koren, Robert Bell, and Chris Volinsky.  Matrix factorization techniques for recommender systems.  _Computer_ , 42(8):30–37, 2009.
  * Li et al. [2023a] Haoxuan Li, Quanyu Dai, Yuru Li, Yan Lyu, Zhenhua Dong, Xiao-Hua Zhou, and Peng Wu.  Multiple robust learning for recommendation.  In _AAAI_ , 2023a.
  * Li et al. [2023b] Haoxuan Li, Yan Lyu, Chunyuan Zheng, and Peng Wu.  TDR-CL: Targeted doubly robust collaborative learning for debiased recommendations.  In _ICLR_ , 2023b.
  * Li et al. [2023c] Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, and Peng Wu.  Balancing unobserved confounding with a few unbiased ratings in debiased recommendations.  In _WWW_ , 2023c.
  * Li et al. [2023d] Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, Peng Wu, and Peng Cui.  Propensity matters: Measuring and enhancing balancing for recommendation.  In _ICML_ , 2023d.
  * Li et al. [2023e] Haoxuan Li, Chunyuan Zheng, and Peng Wu.  StableDR: Stabilized doubly robust learning for recommendation on data missing not at random.  In _ICLR_ , 2023e.
  * Li et al. [2023f] Haoxuan Li, Chunyuan Zheng, Yanghao Xiao, Hao Wang, Fuli Feng, Xiangnan He, Zhi Geng, and Peng Wu.  Removing hidden confounding in recommendation: A unified multi-task learning approach.  In _NeurIPS_ , 2023f.
  * Li et al. [2024] Haoxuan Li, Chunyuan Zheng, Yanghao Xiao, Peng Wu, Zhi Geng, Xu Chen, and Peng Cui.  Debiased collaborative filtering with kernel-based causal balancing.  In _ICLR_ , 2024.
  * Li and Racine [2023] Qi Li and Jeffrey Scott Racine.  _Nonparametric econometrics: theory and practice_.  Princeton University Press, 2023.
  * Liu et al. [2021] Dugang Liu, Pengxiang Cheng, Hong Zhu, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming.  Mitigating confounding bias in recommendation via information bottleneck.  In _RecSys_ , 2021.
  * Luo et al. [2024] Huishi Luo, Fuzhen Zhuang, Ruobing Xie, Hengshu Zhu, Deqing Wang, Zhulin An, and Yongjun Xu.  A survey on causal inference for recommendation.  _The Innovation_ , 2024.
  * Lv et al. [2023] Zheqi Lv, Wenqiao Zhang, Shengyu Zhang, Kun Kuang, Feng Wang, Yongwei Wang, Zhengyu Chen, Tao Shen, Hongxia Yang, Beng Chin Ooi, et al.  Duet: A tuning-free device-cloud collaborative parameters generation framework for efficient device model generalization.  In _WWW_ , 2023.
  * Lv et al. [2024] Zheqi Lv, Wenqiao Zhang, Zhengyu Chen, Shengyu Zhang, and Kun Kuang.  Intelligent model update strategy for sequential recommendation.  In _WWW_ , 2024.
  * Ma et al. [2018] Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai.  Entire space multi-task model: An effective approach for estimating post-click conversion rate.  In _SIGIR_ , 2018.
  * Marlin and Zemel [2009] Benjamin M Marlin and Richard S Zemel.  Collaborative prediction and ranking with non-random missing data.  In _RecSys_ , 2009.
  * Mohri et al. [2018] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.  _Foundations of Machine Learning_.  MIT Press, 2018.
  * Neyman [1990] Jerzy Splawa Neyman.  On the application of probability theory to agricultural experiments. essay on principles. section 9.  _Statistical Science_ , 5:465–472, 1990.
  * Ogburn and VanderWeele [2014] Elizabeth L. Ogburn and Tyler J. VanderWeele.  Causal diagrams for interference.  _Statistical Science_ , 29:559–578, 2014.
  * Ogburn and VanderWeele [2017] Elizabeth L. Ogburn and Tyler J. VanderWeele.  Vaccines, contagion, and social networks.  _The Annals of Applied Statistics_ , 11:919–948, 2017.
  * Rubin [1974] Donald B. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies.  _Journal of Educational Psychology_ , 66:688–701, 1974.
  * Rubin [1980] Donald B. Rubin.  Discussion of randomization analysis of experimental data in the fisher randomization test by basu.  _Journal of the American Statistical Association_ , 75:591–593, 1980.
  * Saito [2020a] Yuta Saito.  Asymmetric tri-training for debiasing missing-not-at-random explicit feedback.  In _SIGIR_ , 2020a.
  * Saito [2020b] Yuta Saito.  Doubly robust estimator for ranking metrics with post-click conversions.  In _RecSys_ , 2020b.
  * Saito and Nomura [2022] Yuta Saito and Masahiro Nomura.  Towards resolving propensity contradiction in offline recommender learning.  In _IJCAI_ , 2022.
  * Saito et al. [2020] Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata.  Unbiased recommender learning from missing-not-at-random implicit feedback.  In _WSDM_ , 2020.
  * Schnabel et al. [2016] Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims.  Recommendations as treatments: Debiasing learning and evaluation.  In _ICML_ , 2016.
  * Shalev-Shwartz and Ben-David [2014] Shai Shalev-Shwartz and Shai Ben-David.  _Understanding Machine Learning: From Theory to Algorithms_.  Cambridge University Press, 2014.
  * Sobel [2006] Michael E Sobel.  What do randomized studies of housing mobility demonstrate? causal inference in the face of interference.  _Journal of the American Statistical Association_ , 101:1398–1407, 2006.
  * Steck [2010] Harald Steck.  Training and testing of recommender systems on data missing not at random.  In _KDD_ , 2010.
  * Swaminathan and Joachims [2015] Adith Swaminathan and Thorsten Joachims.  The self-normalized estimator for counterfactual learning.  In _NeurIPS_ , 2015.
  * Sävje et al. [2021] Fredrik Sävje, Peter M. Aronow, and Peter M. Aronows.  Average treatment effects in the presence of unknown interference.  _Annals of Statistics_ , 49:673–701, 2021.
  * Tchetgen and VanderWeele [2012] Eric J. Tchetgen Tchetgen and Tyler J. VanderWeele.  On causal inference in the presence of interference.  _Statistical Methods in Medical Research_ , 21:55–75, 2012.
  * Tchetgen et al. [2021] Eric J. Tchetgen Tchetgen, Isabel R. Fulcher, and Ilya Shpitser.  Auto-g-computation of causal effects on a network.  _Journal of the American Statistical Association_ , 116:833–844, 2021.
  * Wang et al. [2022a] Hao Wang, Tai-Wei Chang, Tianqiao Liu, Jianmin Huang, Zhichao Chen, Chao Yu, Ruopeng Li, and Wei Chu.  ESCM2: Entire space counterfactual multi-task model for post-click conversion rate estimation.  In _SIGIR_ , 2022a.
  * Wang et al. [2023a] Hao Wang, Jiajun Fan, Zhichao Chen, Haoxuan Li, Weiming Liu, Tianqiao Liu, Quanyu Dai, Yichao Wang, Zhenhua Dong, and Ruiming Tang.  Optimal transport for treatment effect estimation.  In _NeurIPS_ , 2023a.
  * Wang et al. [2022b] Haotian Wang, Wenjing Yang, Longqi Yang, Anpeng Wu, Liyang Xu, Jing Ren, Fei Wu, and Kun Kuang.  Estimating individualized causal effect with confounded instruments.  In _KDD_ , 2022b.
  * Wang et al. [2023b] Haotian Wang, Kun Kuang, Haoang Chi, Longqi Yang, Mingyang Geng, Wanrong Huang, and Wenjing Yang.  Treatment effect estimation with adjustment feature selection.  In _KDD_ , 2023b.
  * Wang et al. [2024] Haotian Wang, Kun Kuang, Long Lan, Zige Wang, Wanrong Huang, Fei Wu, and Wenjing Yang.  Out-of-distribution generalization with causal feature separation.  _IEEE Transactions on Knowledge and Data Engineering_ , 36(4):1758–1772, 2024.
  * Wang et al. [2023c] Jun Wang, Haoxuan Li, Chi Zhang, Dongxu Liang, Enyun Yu, Wenwu Ou, and Wenjia Wang.  CounterCLR: Counterfactual contrastive learning with non-random missing data in recommendation.  In _ICDM_ , 2023c.
  * Wang et al. [2023d] Wenjie Wang, Yang Zhang, Haoxuan Li, Peng Wu, Fuli Feng, and Xiangnan He.  Causal recommendation: Progresses and future directions.  In _SIGIR_ , 2023d.
  * Wang et al. [2019] Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi.  Doubly robust joint learning for recommendation on data missing not at random.  In _ICML_ , 2019.
  * Wang et al. [2021] Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi.  Combating selection biases in recommender systems with a few unbiased ratings.  In _WSDM_ , 2021.
  * Wang et al. [2020] Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan E. Kuruoglu, and Yefeng Zheng.  Information theoretic counterfactual learning from missing-not-at-random feedback.  In _NeurIPS_ , 2020.
  * Wu et al. [2022] Peng Wu, Haoxuan Li, Yuhao Deng, Wenjie Hu, Quanyu Dai, Zhenhua Dong, Jie Sun, Rui Zhang, and Xiao-Hua Zhou.  On the opportunity of causal learning in recommendation systems: Foundation, estimation, prediction and challenges.  In _IJCAI_ , 2022.
  * Wu et al. [2024] Peng Wu, ShaSha Han, Xingwei Tong, and Runze Li.  Propensity score regression for causal inference with treatment heterogeneity.  _Statistica Sinica_ , 34:747–769, 2024.
  * Zhang et al. [2023] Min Zhang, Junkun Yuan, Yue He, Wenbin Li, Zhengyu Chen, and Kun Kuang.  Map: Towards balanced generalization of iid and ood through model-agnostic adapters.  In _ICCV_ , 2023.
  * Zhang et al. [2024] Min Zhang, Haoxuan Li, Fei Wu, and Kun Kuang.  MetaCoCo: A new few-shot classification benchmark with spurious correlation.  In _ICLR_ , 2024.
  * Zhang et al. [2020] Wenhao Zhang, Wentian Bao, Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani.  Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning.  In _WWW_ , 2020.
  * Zheng et al. [2021] Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Depeng Jin, and Yong Li.  Disentangling user interest and conformity for recommendation with causal embedding.  In _WWW_ , 2021.
  * Zou et al. [2023] Hao Zou, Haotian Wang, Renzhe Xu, Bo Li, Jian Pei, Ye Jun Jian, and Peng Cui.  Factual observation based heterogeneity learning for counterfactual prediction.  In _CLeaR_ , 2023.

##  Appendix A Related Work

Selection Bias in Recommender System. Recommender system (RS) plays an
important role in the current era of information explosion [Huang et al.,
2023, Lv et al., 2023, 2024]. However, due to the self-selection by the users,
the data collected by RS will often contain selection bias, which will result
in the distribution of training data (observed population) different from that
of test data (target population), leading to the challenges of achieving
unbiased learning [Wang et al., 2022b, 2023b, Zou et al., 2023, Wang et al.,
2023a, 2024]. If we learn the model directly on the training data without
debiasing, it will result in a sub-optimal performance on the test data [Wang
et al., 2023c, Zhang et al., 2023, Bai et al., 2024, Zhang et al., 2024]. Many
methods have been developed to address selection bias [Wu et al., 2022, Wang
et al., 2023d, Luo et al., 2024], for instance, Marlin and Zemel [2009], Steck
[2010] discussed the error-imputation-based methods, Schnabel et al. [2016]
recommended using the inverse propensity scoring (IPS) method for unbiased
learning. Wang et al. [2019] proposed a doubly robust (DR) joint learning
method and achieved superior performance. Subsequently, various novel model
structures and algorithms are designed to enhance the DR method, such as Guo
et al. [2021], Dai et al. [2022], Li et al. [2024], which proposed new DR
methods by further reducing the bias or variance of the DR estimator. Zhang et
al. [2020] proposed multi-task learning through parameter sharing between the
propensity and prediction model, Chen et al. [2021a], Wang et al. [2021], Li
et al. [2023c] proposed using a small uniform dataset to enhance the
performance of prediction models, and Ding et al. [2022] proposed an
adversarial learning-based framework to address the unmeasured confounders.
However, a user’s feedback on an item may receive influence from the other
user-item pairs [Zheng et al., 2021]. To fill this gap, Chen et al. [2021b]
focuses on the task of learning to rank (LTR), addressing position bias using
implicit feedback data. They consider other user-item interactions as
confounders from a counterfactual perspective and use embedding as a proxy
confounder to capture the influence of other user-item interactions. Different
from Chen et al. [2021b], our paper focuses on the selection bias in the
context of rating prediction, and regards other user-item interactions as a
new treatment from the perspective of interference in causal inference. We
formulate the influence of other user-item interactions as an interference
problem in causal inference and introduce a treatment representation to
capture the influence. On this basis, we propose a novel ideal loss that can
be used to deal with selection bias in the presence of interference.

Interference in Causal Inference. Interference is a common problem in
observational studies, and the effects of interference are also called
spillover effects in economics or peer effects in social sciences [Forastiere
et al., 2021]. Early literature focuses on the case of partial interference
[Hong and Raudenbush, 2006, Sobel, 2006, Hudgens and Halloran, 2008, Tchetgen
and VanderWeele, 2012, Ferracci et al., 2014], i.e., the sample can be divided
into multiple groups, with interference between units in the same group, while
units between groups are independent. Recent works have attempted to further
relax the partial interference assumption by allowing for a wide variety of
interference patterns [Ogburn and VanderWeele, 2014, Aronow and Samii, 2017],
such as direct interference [Forastiere et al., 2021], interference by
contagion [Ogburn and VanderWeele, 2017], allocational interference [Ogburn
and VanderWeele, 2014], or their hybrids [Tchetgen et al., 2021]. These
studies differ from ours in several important ways: (1) their goal is to
estimate the main effect and neighborhood effect, while our goal is to achieve
unbiased learning of the prediction model that is more challenging and need to
carefully design the loss as well as the training algorithm to mitigate the
neighborhood effect; (2) they do not use treatment representation and the
estimation does not take into account the possible cases of continuous and
multi-dimensional representations.

##  Appendix B Proofs

###  B.1 Proofs of Theorems 1 and 2

Recall that
$p_{u,i}(\bm{g})=\mathbb{P}(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})$ and
$\hat{r}_{u,i}=f_{\theta}(x_{u,i})$ are functions of $x_{u,i}$,
$\delta_{u,i}(\bm{g})=\delta(\hat{r}_{u,i},r_{u,i}(1,\bm{g}))$, $\mathbb{P}$
and $\mathbb{E}$ denote the distribution and expectation on the target
population $\mathcal{D}$, and $p(\cdot)$ denotes the probability density
function of $\mathbb{P}$.

######  Theorem 1 (Identifiability).

Under Assumptions 1–3, $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})$ and $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$ are
identifiable.

###### Proof of Theorem 1.

Since

$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})=\int\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g},$

it suffices to show that $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})$ is identifiable. This follows immediately from the following
equations

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})=\mathbb{E}[\delta(\hat{r}_{u,i},r_{u,i}(1,\bm{g}))]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta(\hat{r}_{u,i},r_{u,i}(1,\bm{g}))\mid
x_{u,i}\Big{\\}}\Big{]}\qquad\text{(the law of iterated expectations)}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta(\hat{r}_{u,i},r_{u,i}(1,\bm{g}))\mid
x_{u,i},o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\Big{\\}}\Big{]}\qquad\text{(Assumption
3)}$ $\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta(\hat{r}_{u,i},r_{u,i})\mid
x_{u,i},o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\Big{\\}}\Big{]}\qquad\text{(Assumption
2)}$ $\displaystyle={}$
$\displaystyle\int\int\delta(\hat{r}_{u,i},r_{u,i})p(r_{u,i}|x_{u,i},o_{u,i}=1,\bm{g}_{u,i}=\bm{g})p(x_{u,i})dr_{u,i}dx_{u,i}.$

∎

######  Theorem 2 (Link to Selection Bias).

Under Assumptions 1–3,

(a) if $\bm{g}_{u,i}\perp\\!\\!\\!\perp o_{u,i}\mid x_{u,i}$,
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})=\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}});$

(b) if $\bm{g}_{u,i}\not\\!\perp\\!\\!\\!\perp o_{u,i}\mid x_{u,i}$,
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})-\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$ equals

$\displaystyle\int\mathbb{E}\Big{[}\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\cdot\Big{\\{}p(\bm{g}_{u,i}=\bm{g}|x_{u,i})-p(\bm{g}_{u,i}=\bm{g}|x_{u,i},o_{u,i}=1)\Big{\\}}\Big{]}\pi(\bm{g})d\bm{g}.$

###### Proof of Theorem 2.

For previous methods addressing selection bias without taking into account
interference, the ideal loss $\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$
is

$\displaystyle\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})={}$
$\displaystyle\mathbb{E}[\delta(\hat{r}_{u,i},r_{u,i}(1))]$ $\displaystyle={}$
$\displaystyle\mathbb{E}[\mathbb{E}\\{\delta(\hat{r}_{u,i},r_{u,i}(1))|x_{u,i}\\}]$
$\displaystyle={}$
$\displaystyle\mathbb{E}[\mathbb{E}\\{\delta(\hat{r}_{u,i},r_{u,i})|x_{u,i},o_{u,i}=1\\}]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta(\hat{r}_{u,i},r_{u,i})|x_{u,i},o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\Big{\\}}\cdot
p(\bm{g}_{u,i}=\bm{g}|x_{u,i},o_{u,i}=1)\Big{]}$ $\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta_{u,i}(\bm{g})|x_{u,i},o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\Big{\\}}\cdot
p(\bm{g}_{u,i}=\bm{g}|x_{u,i},o_{u,i}=1)\Big{]}$ $\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta_{u,i}(\bm{g})|x_{u,i}\Big{\\}}\cdot
p(\bm{g}_{u,i}=\bm{g}|x_{u,i},o_{u,i}=1)\Big{]}.$

For the proposed method addressing selection bias under interference, our
newly defined ideal loss $\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})$ is

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$
$\displaystyle={}\int\mathbb{E}[\delta(\hat{r}_{u,i},r_{u,i}(1,\bm{g}))]\pi(\bm{g})d\bm{g}$
$\displaystyle={}\int\mathbb{E}[\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}]\pi(\bm{g})d\bm{g}$
$\displaystyle={}\int\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta_{u,i}(\bm{g})|x_{u,i},\bm{g}_{u,i}=\bm{g}\Big{\\}}\cdot
p(\bm{g}_{u,i}=\bm{g}\mid x_{u,i})\Big{]}\pi(\bm{g})d\bm{g}$
$\displaystyle={}\int\mathbb{E}\Big{[}\mathbb{E}\Big{\\{}\delta_{u,i}(\bm{g})|x_{u,i}\Big{\\}}\cdot
p(\bm{g}_{u,i}=\bm{g}\mid x_{u,i})\Big{]}\pi(\bm{g})d\bm{g}.$

Theorem 2(b) follows immediately from these two rewritten equations. When
$\bm{g}_{u,i}\perp\\!\\!\\!\perp o_{u,i}\mid x_{u,i}$, we have
$p(\bm{g}_{u,i}=\bm{g}\mid x_{u,i}=x,o_{u,i}=1)=p(\bm{g}_{u,i}=\bm{g}\mid
x_{u,i}=x)$, which leads to
$\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})=\mathcal{L}_{\mathrm{ideal}}(\mathbf{\hat{R}})$. This completes the proof
of Theorem 2(a).

∎

###  B.2 Proof of Theorem 3

######  Theorem 3 (Bias and Variance of N-IPS and N-DR).

Under Assumptions 1–4,

(a) the bias of the N-DR estimator is given as

$\text{Bias}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2}),$

where $\mu_{2}=\int K(t)t^{2}dt$. The bias of the N-IPS estimator is given as

$\text{Bias}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2});$

(b) the variance of the N-DR estimator is given as

$\text{Var}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{|\mathcal{D}|h}\int\psi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}),$

where

$\psi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\\{\delta_{u,i}(\bm{g}^{\prime})-\hat{\delta}_{u,i}(\bm{g}^{\prime})\\}\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$, $\bar{K}(\cdot)=\int
K\left(t\right)K\left(\cdot+t\right)dt$. The variance of the N-IPS estimator
is given as

$\displaystyle\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))={}$
$\displaystyle\frac{1}{|\mathcal{D}|h}\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}),$

where

$\varphi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$.

###### Proof of Theorem 3.

(a) We first show the bias of the N-IPS estimator, and the bias of the N-DR
estimator can be shown similarly. For a given $\bm{g}$, we have

$\displaystyle\mathbb{E}[\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})]={}$ $\displaystyle\mathbb{E}\left[\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot
p_{u,i}(\bm{g})}\right]$ $\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\mathbb{E}\Big{\\{}o_{u,i}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\Big{|}x_{u,i}\Big{\\}}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int\int
o_{u,i}\frac{1}{h}K(\frac{\bm{g}_{u,i}-\bm{g}}{h})p(o_{u,i},\bm{g}_{u,i}|x_{u,i})do_{u,i}d\bm{g}_{u,i}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int\frac{1}{h}K(\frac{\bm{g}_{u,i}-\bm{g}}{h})p(o_{u,i}=1,\bm{g}_{u,i}|x_{u,i})d\bm{g}_{u,i}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right],$

where the second equation follows from Assumption 3. Letting
$t=(\bm{g}_{u,i}-\bm{g})/h$, we have $\bm{g}_{u,i}=\bm{g}+ht$ and
$d\bm{g}_{u,i}=hdt$, and then

$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int\frac{1}{h}K(\frac{\bm{g}_{u,i}-\bm{g}}{h})p(o_{u,i}=1,\bm{g}_{u,i}|x_{u,i})d\bm{g}_{u,i}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$ $\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int
K(t)p(o_{u,i}=1,\bm{g}+ht|x_{u,i})dt\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\frac{1}{p_{u,i}(\bm{g})}\int
K(t)\Big{\\{}p(o_{u,i}=1,\bm{g}|x_{u,i})+\frac{\partial
p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}}ht$
$\displaystyle+\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\frac{h^{2}t^{2}}{2}+o(h^{2})\Big{\\}}dt\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\Big{]}$
$\displaystyle={}$ $\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int
K(t)dt\cdot
p(o_{u,i}=1,\bm{g}|x_{u,i})\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle+\mathbb{E}\Big{[}\frac{1}{p_{u,i}(\bm{g})}\int
K(t)tdt\cdot\frac{\partial
p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}}h\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\Big{]}$
$\displaystyle+\mathbb{E}\Big{[}\frac{1}{p_{u,i}(\bm{g})}\int
K(t)t^{2}dt\cdot\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\frac{h^{2}}{2}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\Big{]}+o(h^{2})$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]+\frac{1}{2}\mu_{2}\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}h^{2}+o(h^{2})$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\delta_{u,i}(\bm{g})\right]+\frac{1}{2}\mu_{2}\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}h^{2}+o(h^{2})$
$\displaystyle={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})+\frac{1}{2}\mu_{2}\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}h^{2}+o(h^{2}),$

where the third equation is a Taylor expansion of
$p(o_{u,i}=1,\bm{g}+ht|x_{u,i})$ under Assumption 4(a). Thus, the bias of
$\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$ is

$\displaystyle\mathbb{E}[\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})]-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\int_{\bm{g}}\big{\\{}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\big{\\}}\pi(\bm{g})d\bm{g}\right]$ $\displaystyle={}$
$\displaystyle\int_{\bm{g}}\mathbb{E}\big{\\{}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\big{\\}}\pi(\bm{g})d\bm{g}$ $\displaystyle={}$
$\displaystyle\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2}).$

Likewise, for a given $\bm{g}$ and $\hat{\delta}_{u,i}(\bm{g})$, by a similar
argument of proof of the bias of N-IPS estimator,

$\displaystyle\mathbb{E}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})]$ $\displaystyle={}$
$\displaystyle\mathbb{E}\left[\delta_{u,i}(\bm{g})+\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}}{h\cdot
p_{u,i}(\bm{g})}\right]$ $\displaystyle={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})+\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\mathbb{E}\Big{\\{}o_{u,i}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\Big{|}x_{u,i}\Big{\\}}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})+\frac{1}{2}\mu_{2}\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}h^{2}+o(h^{2}).$

Thus, the bias of $\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})$ is
given as

$\displaystyle\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2}).$

(b) By definition, the variance of
$\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$ can be represented as

$\displaystyle\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))$ $\displaystyle={}$
$\displaystyle\text{Var}\left[\int\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g}\right]$ $\displaystyle={}$
$\displaystyle\text{Var}\left[\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right]$
$\displaystyle={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\text{Var}\left[\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right]$
$\displaystyle={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\Biggl{[}\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)^{2}\right\\}$
$\displaystyle-\left\\{\mathbb{E}\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)\right\\}^{2}\Biggr{]}.$
(A.1)

According to the above result of the bias of the N-IPS estimator, we have

$\displaystyle\left\\{\mathbb{E}\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)\right\\}^{2}$
$\displaystyle={}$
$\displaystyle\left\\{\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})+\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2})\right\\}^{2}$ $\displaystyle={}$
$\displaystyle[\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})]^{2}+O(h^{2}).$ (A.2)

Then, we focus on analyzing the following term

$\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)^{2}\right\\}.$

We can observe that

$\displaystyle\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)^{2}$
$\displaystyle={}$
$\displaystyle\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)$
$\displaystyle\cdot\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{h}\right)\cdot\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}\right)$
$\displaystyle={}$
$\displaystyle\int\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{h^{2}}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)K\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{h}\right)\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g})\pi(\bm{g}^{\prime})d\bm{g}d\bm{g}^{\prime},$

then, we swap the order of integration and expectation, which leads to that

$\displaystyle\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)^{2}\right\\}$
$\displaystyle={}$
$\displaystyle\int\int\mathbb{E}\left[\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{h^{2}}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)K\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{h}\right)\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\right]\pi(\bm{g})\pi(\bm{g}^{\prime})d\bm{g}d\bm{g}^{\prime}.$

Let $\bm{g}_{u,i}=\bm{g}+ht$, then
$\bm{g}_{u,i}-\bm{g}^{\prime}=(\bm{g}-\bm{g}^{\prime})+ht$. We have

$\displaystyle\mathbb{E}\left[\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{h^{2}}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)K\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{h}\right)\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\mathbb{E}\left\\{\mathbb{I}(o_{u,i}=1)\frac{1}{h^{2}}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)K\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{h}\right)\Big{|}x_{u,i}\right\\}\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\int\left\\{\frac{1}{h}K\left(t\right)K\left(\frac{\bm{g}-\bm{g}^{\prime}}{h}+t\right)p(o_{u,i}=1,\bm{g}+ht|x_{u,i})\right\\}dt\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\int\left\\{\frac{1}{h}K\left(t\right)K\left(\frac{\bm{g}-\bm{g}^{\prime}}{h}+t\right)p(o_{u,i}=1,\bm{g}|x_{u,i})+O(h)t\right\\}dt\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\int\frac{1}{h}K\left(t\right)K\left(\frac{\bm{g}-\bm{g}^{\prime}}{h}+t\right)dt\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\right]\cdot\\{1+O(h)\\}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\int\frac{1}{h}K\left(t\right)K\left(\frac{\bm{g}-\bm{g}^{\prime}}{h}+t\right)dt\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\right]\cdot\\{1+O(h)\\}.$

Denote $\int
K\left(t\right)K\left(\frac{\bm{g}-\bm{g}^{\prime}}{h}+t\right)dt=\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})$,
then

$\displaystyle\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})}\cdot\frac{1}{h}K\left(\frac{\bm{g}_{u,i}-\bm{g}}{h}\right)\cdot\delta_{u,i}(\bm{g})\pi(\bm{g})d\bm{g}\right)^{2}\right\\}$
$\displaystyle={}$
$\displaystyle\int\int\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{h}\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\right]\cdot\\{1+O(h)\\}.\pi(\bm{g})\pi(\bm{g}^{\prime})d\bm{g}d\bm{g}^{\prime}$
$\displaystyle={}$
$\displaystyle\int\mathbb{E}\left[\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{h}\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}\right]\cdot\\{1+O(h)\\}.\pi(\bm{g})d\bm{g}$
$\displaystyle\triangleq{}$
$\displaystyle\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}\frac{1}{h}+O(1),$ (A.3)

where

$\varphi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$.

Combing equations (A.1), (A.2), and (A.3) gives that

$\displaystyle\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\left[\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}\frac{1}{h}+O(1)-[\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})]^{2}+O(h^{2})\right]$ $\displaystyle={}$
$\displaystyle\frac{1}{|\mathcal{D}|h}\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}).$

Similarly, the variance of the N-DR estimator is given by

$\text{Var}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{|\mathcal{D}|h}\int\psi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}),$

where

$\psi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\\{\delta_{u,i}(\bm{g}^{\prime})-\hat{\delta}_{u,i}(\bm{g}^{\prime})\\}\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$.

∎

###  B.3 Proof of Theorem 4

######  Theorem 4 (Optimal Bandwidth of N-IPS and N-DR).

Under Assumptions 1–4,

(a) the optimal bandwidth for the N-IPS estimator in terms of the asymptotic
mean-squared error is

$h^{*}_{\mathrm{N-IPS}}=\left[\frac{\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}}{4|\mathcal{D}|\left(\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\right)^{2}}\right]^{1/5},$

where $\varphi(\bm{g})$ is defined in Theorem 3;

(b) the optimal bandwidth for the N-DR estimator in terms of the asymptotic
mean-squared error is

$h^{*}_{\mathrm{N-DR}}=\left[\frac{\int\psi(\bm{g})\pi(\bm{g})d\bm{g}}{4|\mathcal{D}|\left(\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}\pi(\bm{g})d\bm{g}\right)^{2}}\right]^{1/5}.$

###### Proof of Theorem 4.

Recall that

$\displaystyle\text{Bias}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))={}$
$\displaystyle\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2}),$
$\displaystyle\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))={}$
$\displaystyle\frac{1}{|\mathcal{D}|h}\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}).$

The mean-squared error of the N-IPS estimator is given as

$\displaystyle\mathbb{E}\big{[}\big{(}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})\big{)}^{2}\big{]}$
$\displaystyle={}$
$\displaystyle(\text{Bias}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})))^{2}+\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R}))$
$\displaystyle={}$
$\displaystyle\left(\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\right)^{2}\cdot
h^{4}+o(h^{4})$
$\displaystyle+\frac{1}{|\mathcal{D}|h}\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|h}).$

Minimizing the leading terms of the above mean-squared error with respect to
$h$ leads to that

$h^{*}_{\mathrm{N-IPS}}=\left[\frac{\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}}{4|\mathcal{D}|\left(\frac{1}{2}\mu_{2}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\pi(\bm{g})d\bm{g}\right)^{2}}\right]^{1/5}=O(|\mathcal{D}|^{-1/5}).$

Similarly, the optimal bandwidth for the N-DR estimator in terms of the
asymptotic mean-squared error can be obtained.

∎

###  B.4 Proof of Theorem 5

######  Lemma 1 (McDiarmid’s Inequality).

Let $X_{1},...,X_{m}\in\mathcal{X}^{m}$ be a set of $m\geq 1$ independent
random variables and assume that there exist $c_{1},...,c_{m}>0$ such that
$f:\mathcal{X}^{m}\to\mathbb{R}$ satisfies the following conditions:

$|f(x_{1},...,x_{i},...,x_{m})-f(x_{1},...,x_{i}^{\prime},...,x_{m})|\leq
c_{i},$

for all $i\in\\{1,2,...,m\\}$ and any points
$x_{1},...,x_{m},x_{i}^{\prime}\in\mathcal{X}$. Let $f(S)$ denote
$f(X_{1},...,X_{m})$, then for all $\epsilon>0$, the following inequalities
hold:

$\displaystyle\mathbb{P}[f(S)-\mathbb{E}\\{f(S)\\}\geq\epsilon]\leq{}$
$\displaystyle\exp\left(-\frac{2\epsilon^{2}}{\sum_{i=1}^{m}c_{i}^{2}}\right)$
$\displaystyle\mathbb{P}[f(S)-\mathbb{E}\\{f(S)\\}\leq-\epsilon]\leq{}$
$\displaystyle\exp\left(-\frac{2\epsilon^{2}}{\sum_{i=1}^{m}c_{i}^{2}}\right).$

###### Proof of Lemma 1.

The proof can be found in Appendix D.3 of Mohri et al. [2018]. ∎

######  Lemma 2.

Under the conditions in Lemma 1, we have with probability at least $1-\eta$,

$|f(S)-\mathbb{E}[f(S)]|\leq\sqrt{\frac{\sum_{i=1}^{m}c_{i}^{2}}{2}\log(\frac{2}{\eta})}.$

In particular, if $c_{i}\leq c$ for all $i\in\\{1,2,...,m\\}$,

$|f(S)-\mathbb{E}[f(S)]|\leq c\sqrt{\frac{m}{2}\log(\frac{2}{\eta})}.$

###### Proof of Lemma 2.

This conclusion follows by letting
$\eta=2\exp\left(-\frac{2\epsilon^{2}}{\sum_{i=1}^{m}c_{i}^{2}}\right)$ in
Lemma 1. ∎

######  Lemma 3 (Rademacher Comparison Lemma).

Let $X\in\mathcal{X}$ be a random variable with distribution $\mathbb{P}$,
$X_{1},...,X_{m}$ be a set of independent copies of $X$, $\mathcal{F}$ be a
class of real-valued functions on $\mathcal{X}$. Then we have

$\mathbb{E}\sup_{f\in\mathcal{G}}\left|\frac{1}{m}\sum_{i=1}^{m}f(X_{i})-\mathbb{E}(f(X_{i}))\right|\leq
2\mathbb{E}\left[\mathbb{E}_{\bm{\sigma}\sim\\{-1,+1\\}^{m}}\sup_{f\in\mathcal{G}}\frac{1}{m}\sum_{i=1}^{m}f(X_{i})\sigma_{i}\right],$

where $\bm{\sigma}=(\sigma_{1},...,\sigma_{m})$ is a Rademacher sequence.

###### Proof of Lemma 3.

See Lemma 26.2 of Shalev-Shwartz and Ben-David [2014]. ∎

Recall that $\mathcal{F}$ is the hypothesis space of prediction matrices
$\hat{\bf R}$ (or prediction model $f_{\theta}$), the Rademacher complexity is

$\mathcal{R}(\mathcal{F})=\mathbb{E}_{\bm{\sigma}\sim\\{-1,+1\\}^{|\mathcal{D}|}}\sup_{f_{\theta}\in\mathcal{F}}\Big{[}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\sigma_{u,i}\delta_{u,i}(\bm{g})\Big{]}.$

######  Lemma 4 (Uniform Tail Bound of N-IPS and N-DR).

Under Assumptions 1–5 and suppose that $K(t)\leq M_{K}$, then we have with
probability at least $1-\eta$,

(a)

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}[\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})]\Big{|}\leq{}$
$\displaystyle\frac{2M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+\frac{5}{2}\frac{M_{p}M_{K}M_{\delta}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})};$

(b)

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})]\Big{|}\leq{}$
$\displaystyle\frac{2M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+\frac{5}{2}\frac{M_{p}M_{K}M_{|\delta-\hat{\delta}|}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}.$

###### Proof of Lemma 4.

We first discuss the uniform tail bound of
$\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$, that is, we want to
show the upper bound of $\sup_{\hat{\bf
R}\in\mathcal{F}}\big{|}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}\big{|}$.

Note that

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}\Big{|}$ $\displaystyle={}$ $\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\int\pi(\bm{g})\Biggl{[}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{]}d\bm{g}\Biggr{|}$
$\displaystyle\leq{}$ $\displaystyle\int\pi(\bm{g})\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}d\bm{g}.$

For all prediction model $\hat{\bf R}\in\mathcal{F}$ and $\bm{g}$, we have

$\frac{1}{|\mathcal{D}|}\left|\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}-\frac{\mathbb{I}(o_{u^{\prime},i^{\prime}}=1)\cdot
K\left((\bm{g}_{u^{\prime},i^{\prime}}-\bm{g})/h\right)\cdot\delta_{u^{\prime},i^{\prime}}(\bm{g})}{h\cdot\hat{p}_{u^{\prime},i^{\prime}}(\bm{g})}\right|\leq\frac{M_{p}M_{\delta}M_{K}}{h|\mathcal{D}|},$

then applying Lemma 2 yields that with probability at least $1-\eta/2$

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}$
$\displaystyle\leq{}$ $\displaystyle\mathbb{E}\Biggl{[}\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}\Biggr{]}+\frac{M_{p}M_{\delta}M_{K}}{2h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}$
$\displaystyle\leq{}$
$\displaystyle\frac{M_{p}M_{K}}{h}2\mathbb{E}[\mathcal{R}(\mathcal{F})]+\frac{M_{p}M_{\delta}M_{K}}{2h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})},$
(A.4)

where the last inequality holds by Lemma 3 and

$\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)}{h\cdot\hat{p}_{u,i}(\bm{g})}\leq\frac{M_{p}M_{K}}{h}.$

Recall that

$\mathcal{R}(\mathcal{F})=\mathbb{E}_{\bm{\sigma}\sim\\{-1,+1\\}^{|\mathcal{D}|}}\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{[}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\sigma_{u,i}\delta_{u,i}(\bm{g})\Big{]}$

and let $f(S)=\mathcal{R}(\mathcal{F})$ and $c=2M_{\delta}/|\mathcal{D}|$ in
Lemmas 1 and 2, by applying Lemma 2 again, we have with probability at least
$1-\eta/2$

$\mathbb{E}[\mathcal{R}(\mathcal{F})]-\mathcal{R}(\mathcal{F})\leq
M_{\delta}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}.$ (A.5)

Combining inequalities (A.4) and (A.5) leads to that with probability at least
$1-\eta$

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\delta_{u,i}(\bm{g})}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}$
$\displaystyle\leq{}$
$\displaystyle\frac{2M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+\frac{5}{2}\frac{M_{p}M_{K}M_{\delta}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})},$

which implies Lemma 4(a) by noting that $\int\pi(\bm{g})d\bm{g}=1$.

Next, we show the uniform tail bound of
$\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})$. Note that

$\displaystyle\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})=\delta_{u,i}(\bm{g})+\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}}{h\cdot\hat{p}_{u,i}(\bm{g})}$

and

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}\\{\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})\\}\Big{|}$ $\displaystyle={}$ $\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\int\pi(\bm{g})\Biggl{[}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}}{h\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{]}d\bm{g},$

which has the same form as $\sup_{\hat{\bf
R}\in\mathcal{F}}\big{|}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}\big{|}$, except that $\delta_{u,i}(\bm{g})$ is replaced by
$\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}$. By a similar argument of the proof
of the N-IPS estimator, we obtain the Lemma 4(b).

∎

Let

$\hat{\bf R}^{\dagger}=\arg\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}),\quad\hat{\bf R}^{\ddagger}=\arg\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R}).$

The following Theorem 5 shows the generalization error bounds of the N-IPS and
N-DR estimators.

######  Theorem 5 (Generalization Error Bounds of N-IPS and N-DR).

Under Assumptions 1–5 and suppose that $K(t)\leq M_{K}$, we have with
probability at least $1-\eta$,

(a)

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})\leq{}$ $\displaystyle\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})+\mu_{2}M_{|\delta-\hat{\delta}|}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2})$
$\displaystyle+\frac{4M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{K}M_{|\delta-\hat{\delta}|}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})};$

(b)

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\ddagger})\leq{}$ $\displaystyle\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})+\mu_{2}M_{\delta}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\cdot
h^{2}+o(h^{2})$
$\displaystyle+\frac{4M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{K}M_{\delta}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}.$

###### Proof of Theorem 5.

It suffices to show Theorem 5(a), since Theorem 5(b) can be derived from a
similar argument. Define

$\hat{\bf R}^{*}=\arg\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R}),$

then we have

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R})$
$\displaystyle={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R}^{*})$
$\displaystyle\leq{}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})+\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{*})+\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{*})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R}^{*})$
$\displaystyle\leq{}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})+\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{*})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R}^{*})$
$\displaystyle:={}$ $\displaystyle A+B,$

where

$\displaystyle A={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R}^{\dagger}),$
$\displaystyle B={}$
$\displaystyle\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{*})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf R}^{*}).$

The first term can be decomposed as follows

$\displaystyle A={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})-\mathbb{E}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})]+\mathbb{E}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})]-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R}^{\dagger})$
$\displaystyle={}$
$\displaystyle\left|\text{Bias}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})]\right|+\mathbb{E}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})]-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R}^{\dagger})$
$\displaystyle\leq{}$
$\displaystyle\left|\text{Bias}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})]\right|+\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{[}\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}-\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})\Big{]}$
$\displaystyle={}$
$\displaystyle\frac{1}{2}\mu_{2}\left|\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}\pi(\bm{g})d\bm{g}\right|\cdot
h^{2}+o(h^{2})$ $\displaystyle+\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{[}\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}-\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})\Big{]}$
$\displaystyle\leq{}$
$\displaystyle\frac{1}{2}\mu_{2}M_{|\delta-\hat{\delta}|}\left|\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\right|\cdot
h^{2}+o(h^{2})$ $\displaystyle+\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{[}\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}-\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})\Big{]},$

the upper bound does not depend on $\hat{\bf R}^{\dagger}$. Likewise, the
upper bound of the second term is

$\displaystyle B\leq{}$
$\displaystyle\frac{1}{2}\mu_{2}M_{|\delta-\hat{\delta}|}\left|\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\right|\cdot
h^{2}+o(h^{2})$ $\displaystyle+\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{[}\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}-\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})\Big{]}.$

Then by Lemma 4, we have with probability at least $1-\eta$,

$\displaystyle A+B\leq{}$
$\displaystyle\mu_{2}M_{|\delta-\hat{\delta}|}\left|\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\right|\cdot
h^{2}+o(h^{2})$ $\displaystyle+2\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{[}\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}-\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})\Big{]}$
$\displaystyle\leq{}$
$\displaystyle\mu_{2}M_{|\delta-\hat{\delta}|}\left|\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial\bm{g}^{2}}\Big{]}\pi(\bm{g})d\bm{g}\right|\cdot
h^{2}+o(h^{2})$
$\displaystyle+\frac{4M_{p}M_{K}}{h}\mathcal{R}(\mathcal{F})+5\frac{M_{p}M_{K}M_{|\delta-\hat{\delta}|}}{h}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})},$

which implies the conclusion of Theorem 5(a).

∎

##  Appendix C Extension: Multi-dimensional Treatment Representation

For ease of presentation, we focus on the case of univariate $\bm{g}$ in the
manuscript. In this section, we extend the univariate case and consider the
case of multi-dimensional $\bm{g}$.

Suppose that $\bm{g}$ is a $q$-dimensional vector denoted as
$\bm{g}=(g_{1},...,g_{q})$. In this case, the bandwidth is
$\bm{h}=(h_{1},...,h_{q})$ and the kernel function is
$\bm{K}((\bm{g}_{u,i}-\bm{g})/\bm{h})=\prod_{s=1}^{q}K((g_{u,i}^{s}-g_{s})/h_{s})$,
where $\bm{g}_{u,i}=(g_{u,i}^{1},...,g_{u,i}^{q})$ with $g_{u,i}^{s}$ being
its $s$-th element, $K(\cdot)$ is the univariate kernel function such as
Epanechnikov kernel $K(t)=\frac{3}{4}(1-t^{2})\mathbb{I}\\{|t|\leq 1\\}$ and
Gaussian kernel $K(t)=\frac{1}{\sqrt{2\pi}}\cdot\exp\\{-\frac{t^{2}}{2}\\}$
for $t\in\mathbb{R}$. The N-IPS and N-DR estimators are given as

$\displaystyle\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})={}$
$\displaystyle\int\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g},$
$\displaystyle\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})={}$
$\displaystyle\int\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\pi(\bm{g})d\bm{g},$

where

$\displaystyle\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R}|\bm{g})={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})},$
$\displaystyle\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R}|\bm{g})={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left[\hat{\delta}_{u,i}(\bm{g})+\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\right].$

We show the theoretical properties of the proposed N-IPS and N-DR estimators,
extending the results of Theorems 3–5. First, we present the bias of the N-IPS
and N-DR estimators in the setting of multi-dimension treatment
representation.

######  Assumption 6 (Regularity Conditions for Kernel Smoothing).

(a) For $s=1,...,q$, $h_{s}\to 0$ as $|\mathcal{D}|\to\infty$; (b)
$|\mathcal{D}|\prod_{s=1}^{q}h_{s}\to\infty$ as $|\mathcal{D}|\to\infty$; (c)
$p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}\mid x_{u,i})$ is twice differentiable with
respect to $\bm{g}=(g_{1},...,g_{q})$.

######  Theorem 6 (Bias and Variance of N-IPS and N-DR).

Under Assumptions 1–3 and 6,

(a) the bias of the N-IPS estimator is given as

$\frac{1}{2}\mu_{2}\left[\sum_{s=1}^{q}h_{s}^{2}\int\mathbb{E}\Big{\\{}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{\\}}\pi(\bm{g})d\bm{g}\right]+o(\sum_{s=1}^{q}h_{s}^{2}),$

where $\mu_{2}=\int K(t)t^{2}dt$. The bias of the N-DR estimator is given as

$\frac{1}{2}\mu_{2}\left[\sum_{s=1}^{q}h_{s}^{2}\int\mathbb{E}\Big{\\{}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\big{(}\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\big{)}\Big{\\}}\pi(\bm{g})d\bm{g}\right]+o(\sum_{s=1}^{q}h_{s}^{2});$

(b) the variance of the N-IPS estimator is

$\displaystyle\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))={}$
$\displaystyle\frac{1}{|\mathcal{D}|\prod_{s=1}^{q}h_{s}}\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|\prod_{s=1}^{q}h_{s}}),$

where

$\varphi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$,
$\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{\bm{h}})=\int\prod_{s=1}^{q}K\left(t_{s}\right)K\left(\frac{\bm{g}_{s}-\bm{g}_{s}^{\prime}}{h_{s}}+t_{s}\right)dt_{1}\cdots
dt_{q}$. The variance of the N-DR estimator is

$\text{Var}(\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R}))=\frac{1}{|\mathcal{D}|\prod_{s=1}^{q}h_{s}}\int\psi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|\prod_{s=1}^{q}h_{s}}),$

where

$\psi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\\{\delta_{u,i}(\bm{g}^{\prime})-\hat{\delta}_{u,i}(\bm{g}^{\prime})\\}\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$.

###### Proof of Theorem 6.

We show the bias and variance of the N-IPS estimator, and the bias and
variance of the N-DR estimator can be obtained similarly.

(a) For a given $\bm{g}$, by a similar argument of the proof of Theorem 3(a),

$\displaystyle\mathbb{E}[\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})]={}$
$\displaystyle\mathbb{E}\left[\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\right]$ $\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int\prod_{s=1}^{q}\frac{1}{h_{s}}K(\frac{g_{u,i}^{s}-g_{s}}{h_{s}})p(o_{u,i}=1,\bm{g}_{u,i}|x_{u,i})d\bm{g}_{u,i}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right].$

Let $t_{s}=(g_{u,i}^{s}-g_{s})/h_{s}$ for $s=1,...,q$, then
$g_{u,i}^{s}=g_{s}+h_{s}t_{s}$, $d\bm{g}_{u,i}=dg_{u,i}^{1}\cdots
dg_{u,i}^{q}=\prod_{s=1}^{q}h_{s}dt_{1}\cdots dt_{q}$. By a Taylor expansion
of $p(o_{u,i}=1,g_{1}+h_{1}t_{1},...,g_{q}+h_{q}t_{q}|x_{u,i})$, we have

$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int\prod_{s=1}^{q}\frac{1}{h_{s}}K(\frac{g_{u,i}^{s}-g_{s}}{h_{s}})p(o_{u,i}=1,\bm{g}_{u,i}|x_{u,i})d\bm{g}_{u,i}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\int\cdots\int\prod_{s=1}^{q}K(t_{s})p(o_{u,i}=1,g_{1}+h_{1}t_{1},...,g_{q}+h_{q}t_{q}|x_{u,i})dt_{1}\cdots
dt_{q}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Big{[}\frac{1}{p_{u,i}(\bm{g})}\int\cdots\int\prod_{s=1}^{q}K(t_{s})\Big{\\{}p(o_{u,i}=1,\bm{g}|x_{u,i})+\sum_{s=1}^{q}\frac{\partial
p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial g_{s}}h_{s}t_{s}$
$\displaystyle+\sum_{s=1}^{q}\sum_{s^{\prime}=1}^{q}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial
g_{s}\partial
g_{s^{\prime}}}\frac{h_{s}h_{s^{\prime}}t_{s}t_{s^{\prime}}}{2}+o(\sum_{s=1}^{q}h_{s}^{2})\Big{\\}}dt_{1}\cdots
dt_{q}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\Big{]}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g})}\cdot\prod_{s=1}^{q}\int
K(t_{s})dt_{s}\cdot
p(o_{u,i}=1,\bm{g}|x_{u,i})\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]+0$
$\displaystyle+\mathbb{E}\Big{[}\frac{1}{p_{u,i}(\bm{g})}\sum_{s=1}^{q}\int
K(t_{s})t_{s}^{2}dt_{s}\cdot\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\frac{h_{s}^{2}}{2}\cdot\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\Big{]}+o(\sum_{s=1}^{q}h_{s}^{2})$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\mathbb{E}\\{\delta_{u,i}(\bm{g})|x_{u,i}\\}\right]+\frac{1}{2}\mu_{2}\mathbb{E}\Big{[}\sum_{s=1}^{q}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot
h_{s}^{2}\cdot\delta_{u,i}(\bm{g})\Big{]}+o(\sum_{s=1}^{q}h_{s}^{2})$
$\displaystyle={}$
$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})+\frac{1}{2}\mu_{2}\mathbb{E}\Big{[}\sum_{s=1}^{q}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot
h_{s}^{2}\cdot\delta_{u,i}(\bm{g})\Big{]}+o(\sum_{s=1}^{q}h_{s}^{2}).$

Thus, the bias of $\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$ is

$\displaystyle\mathbb{E}[\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})]-$ $\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})={}\int_{\bm{g}}\mathbb{E}\big{\\{}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})-\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}|\bm{g})\big{\\}}\pi(\bm{g})d\bm{g}$ $\displaystyle={}$
$\displaystyle\frac{1}{2}\mu_{2}\left[\sum_{s=1}^{q}h_{s}^{2}\int\mathbb{E}\Big{\\{}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{\\}}\pi(\bm{g})d\bm{g}\right]+o(\sum_{s=1}^{q}h_{s}^{2}).$

(b) The variance of $\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$ can
be represented as

$\displaystyle\text{Var}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\text{Var}\left[\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right]$ $\displaystyle={}$
$\displaystyle\frac{1}{|\mathcal{D}|}\Biggl{[}\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right)^{2}\right\\}$
$\displaystyle-\left\\{\mathbb{E}\left(\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right)\right\\}^{2}\Biggr{]}.$

By the bias of the N-IPS estimator,

$\displaystyle\left\\{\mathbb{E}\left(\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right)\right\\}^{2}=[\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})]^{2}+O(\sum_{s=1}^{q}h_{s}^{2})=O(1).$

On the other hand, note that

$\displaystyle\left(\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right)^{2}$ $\displaystyle={}$
$\displaystyle\int\int\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{\prod_{s=1}^{q}h_{s}^{2}}\bm{K}\left(\frac{\bm{g}_{u,i}-\bm{g}}{\bm{h}}\right)\bm{K}\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{\bm{h}}\right)\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g})\pi(\bm{g}^{\prime})d\bm{g}d\bm{g}^{\prime},$

we swap the order of integration and expectation, which leads to that

$\displaystyle\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right)^{2}\right\\}$ $\displaystyle={}$
$\displaystyle\int\int\mathbb{E}\left[\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{\prod_{s=1}^{q}h_{s}^{2}}\bm{K}\left(\frac{\bm{g}_{u,i}-\bm{g}}{\bm{h}}\right)\bm{K}\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{\bm{h}}\right)\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\right]\pi(\bm{g})\pi(\bm{g}^{\prime})d\bm{g}d\bm{g}^{\prime}.$

Let $\bm{g}_{u,i}=\bm{g}+\bm{h}\bm{t}$, i.e., $g_{u,i}^{s}=g_{s}+h_{s}t_{s}$,
then $\bm{g}_{u,i}-\bm{g}^{\prime}=(\bm{g}-\bm{g}^{\prime})+\bm{ht}$, we have

$\displaystyle\mathbb{E}\Biggl{[}\frac{\mathbb{I}(o_{u,i}=1)}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{\prod_{s=1}^{q}h_{s}^{2}}\bm{K}\left(\frac{\bm{g}_{u,i}-\bm{g}}{\bm{h}}\right)\bm{K}\left(\frac{\bm{g}_{u,i}-\bm{g}^{\prime}}{\bm{h}}\right)\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Biggr{]}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Biggl{[}\frac{1}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\int\left\\{\prod_{s=1}^{q}\frac{1}{h_{s}^{2}}K\left(t_{s}\right)K\left(\frac{\bm{g}_{s}-\bm{g}_{s}^{\prime}}{h_{s}}+t_{s}\right)p(o_{u,i}=1,\bm{g}+\bm{ht}|x_{u,i})\right\\}dt_{1}\cdots
dt_{q}$
$\displaystyle\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\Biggr{]}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\Biggl{[}\frac{1}{p_{u,i}(\bm{g})p_{u,i}(\bm{g}^{\prime})}\cdot\int\left\\{\prod_{s=1}^{q}\frac{1}{h_{s}^{2}}K\left(t_{s}\right)K\left(\frac{\bm{g}_{s}-\bm{g}_{s}^{\prime}}{h_{s}}+t_{s}\right)p(o_{u,i}=1,\bm{g}|x_{u,i})+O(\sum_{s=1}^{q}h_{s})\right\\}dt_{1}\cdots
dt_{q}$
$\displaystyle\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\Biggr{]}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\int\prod_{s=1}^{q}\frac{1}{h_{s}}K\left(t_{s}\right)K\left(\frac{\bm{g}_{s}-\bm{g}_{s}^{\prime}}{h_{s}}+t_{s}\right)dt_{1}\cdots
dt_{q}\cdot\mathbb{E}\left\\{\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\Big{|}x_{u,i}\right\\}\right]\cdot\\{1+O(\sum_{s=1}^{q}h_{s})\\}$
$\displaystyle={}$
$\displaystyle\mathbb{E}\left[\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\int\prod_{s=1}^{q}\frac{1}{h_{s}}K\left(t_{s}\right)K\left(\frac{\bm{g}_{s}-\bm{g}_{s}^{\prime}}{h_{s}}+t_{s}\right)dt_{1}\cdots
dt_{q}\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\right]\cdot\\{1+O(\sum_{s=1}^{q}h_{s})\\}.$

Thus, define
$\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{\bm{h}})=\int\prod_{s=1}^{q}K\left(t_{s}\right)K\left(\frac{\bm{g}_{s}-\bm{g}_{s}^{\prime}}{h_{s}}+t_{s}\right)dt_{1}\cdots
dt_{q}$, then

$\displaystyle\mathbb{E}\left\\{\left(\int\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot
p_{u,i}(\bm{g})}\pi(\bm{g})d\bm{g}\right)^{2}\right\\}$ $\displaystyle={}$
$\displaystyle\int\mathbb{E}\left[\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\frac{1}{\prod_{s=1}^{q}h_{s}}\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{\bm{h}})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}\right]\cdot\\{1+O(\sum_{s=1}^{q}h_{s})\\}.\pi(\bm{g})d\bm{g}$
$\displaystyle\triangleq{}$
$\displaystyle\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}\frac{1}{\prod_{s=1}^{q}h_{s}}(1+o(1)),$

where

$\varphi(\bm{g})=\int\frac{1}{p_{u,i}(\bm{g}^{\prime})}\cdot\bar{K}(\frac{\bm{g}-\bm{g}^{\prime}}{h})\cdot\delta_{u,i}(\bm{g})\delta_{u,i}(\bm{g}^{\prime})\pi(\bm{g}^{\prime})d\bm{g}^{\prime}$

is a bounded function of $\bm{g}$. Therefore, we have

$\displaystyle\text{Var}(\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}))={}$
$\displaystyle\frac{1}{|\mathcal{D}|\prod_{s=1}^{q}h_{s}}\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}+o(\frac{1}{|\mathcal{D}|\prod_{s=1}^{q}h_{s}}).$

∎

######  Theorem 7 (Optimal Bandwidth of N-IPS and N-DR).

Under Assumptions 1–3 and 6, and assume that $h=h_{1}=\cdots=h_{q}$, then

(a) the optimal bandwidth for the N-IPS estimator in terms of the asymptotic
mean-squared error is

$h^{*}_{\mathrm{N-IPS}}=\left[\frac{\int\varphi(\bm{g})\pi(\bm{g})d\bm{g}}{4|\mathcal{D}|\left(\frac{1}{2}\mu_{2}\sum_{s=1}^{q}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{]}\cdot\pi(\bm{g})d\bm{g}\right)^{2}}\right]^{1/(4+q)},$

where $\varphi(\bm{g})$ is defined in Theorem 6;

(b) the optimal bandwidth for the N-DR estimator in terms of the asymptotic
mean-squared error is

$h^{*}_{\mathrm{N-DR}}=\left[\frac{\int\psi(\bm{g})\pi(\bm{g})d\bm{g}}{4|\mathcal{D}|\left(\frac{1}{2}\mu_{2}\sum_{s=1}^{q}\int\mathbb{E}\Big{[}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\\{\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\\}\Big{]}\cdot\pi(\bm{g})d\bm{g}\right)^{2}}\right]^{1/(4+q)}.$

###### Proof of Theorem 7.

This conclusion can be derived similarly from the proof of Theorem 4.

∎

Then, we obtain the uniform tail bound of the N-IPS and N-DR estimators with
multi-dimensional treatment representation.

######  Lemma 5 (Uniform Tail Bound of N-IPS and N-DR).

Under Assumptions 1–3, 5, and 6, and suppose that $K(t)\leq M_{K}$, then we
have with probability at least $1-\eta$,

(a)

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathbb{E}[\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})]-\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})\Big{|}\leq{}$
$\displaystyle\frac{2M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{\delta}(M_{K})^{q}}{2\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})};$

(b)

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathbb{E}[\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf
R})]-\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})\Big{|}\leq{}$
$\displaystyle\frac{2M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{|\delta-\hat{\delta}|}(M_{K})^{q}}{2\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}.$

###### Proof of Lemma 5.

It is sufficient to prove the uniform tail bound of
$\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$, and the result for
$\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})$ can be derived by a
similar argument.

Note that

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Big{|}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})-\mathbb{E}\\{\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R})\\}\Big{|}$ $\displaystyle\leq{}$
$\displaystyle\int\pi(\bm{g})\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}$
$\displaystyle-\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}d\bm{g}.$

For all prediction model $\hat{\bf R}\in\mathcal{F}$ and $\bm{g}$, we have

$\frac{1}{|\mathcal{D}|}\left|\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}-\frac{\mathbb{I}(o_{u^{\prime},i^{\prime}}=1)\cdot\bm{K}\left((\bm{g}_{u^{\prime},i^{\prime}}-\bm{g})/\bm{h}\right)\cdot\delta_{u^{\prime},i^{\prime}}(f)}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u^{\prime},i^{\prime}}(\bm{g})}\right|\leq\frac{M_{p}M_{\delta}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}|\mathcal{D}|},$

then applying Lemma 2 yields that with probability at least $1-\eta/2$

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}-\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}$
$\displaystyle\leq{}$ $\displaystyle\mathbb{E}\Biggl{[}\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}-\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}\Biggr{]}$
$\displaystyle+\frac{M_{p}M_{\delta}(M_{K})^{q}}{2\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}$
$\displaystyle\leq{}$
$\displaystyle\frac{M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}2\mathbb{E}[\mathcal{R}(\mathcal{F})]+\frac{M_{p}M_{\delta}(M_{K})^{q}}{2\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})},$

where the last inequality holds by Lemma 3 and

$\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\leq\frac{M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}.$

Applying (A.5) yields that with probability at least $1-\eta$

$\displaystyle\sup_{\hat{\bf
R}\in\mathcal{F}}\Biggl{|}\mathbb{E}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}-\frac{1}{|\mathcal{D}|}\sum_{(u,i)\in\mathcal{D}}\left\\{\frac{\mathbb{I}(o_{u,i}=1)\cdot\bm{K}\left((\bm{g}_{u,i}-\bm{g})/\bm{h}\right)\cdot\delta_{u,i}(\bm{g})}{\prod_{s=1}^{q}h_{s}\cdot\hat{p}_{u,i}(\bm{g})}\right\\}\Biggr{|}$
$\displaystyle\leq{}$
$\displaystyle\frac{2M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{\delta}(M_{K})^{q}}{2\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})},$

which implies Lemma 5(a) by noting that $\int\pi(\bm{g})d\bm{g}=1$.

∎

Let

${\bf R}^{\dagger}=\arg\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf
R}),\quad{\bf R}^{\ddagger}=\arg\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R}).$

The following Theorem 8 shows the generalization error bounds of the N-IPS and
N-DR estimators.

######  Theorem 8 (Generalization Error Bounds of N-IPS and N-DR).

Under Assumptions 1–3, 5, and 6, and suppose that $K(t)\leq M_{K}$, then we
have with probability at least $1-\eta$,

(a)

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\dagger})\leq{}$ $\displaystyle\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})+\mu_{2}\left[\sum_{s=1}^{q}h_{s}^{2}\int\mathbb{E}\Big{\\{}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\delta_{u,i}(\bm{g})\Big{\\}}\pi(\bm{g})d\bm{g}\right]$
$\displaystyle+\frac{4M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{\delta}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}+o(\sum_{s=1}^{q}h_{s}^{2});$

(b)

$\displaystyle\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R}^{\ddagger})\leq{}$ $\displaystyle\min_{\hat{\bf
R}\in\mathcal{F}}\mathcal{L}_{\mathrm{ideal}}^{\mathrm{N}}(\hat{\bf
R})+\mu_{2}\left[\sum_{s=1}^{q}h_{s}^{2}\int\mathbb{E}\Big{\\{}\frac{\partial^{2}p(o_{u,i}=1,\bm{g}_{u,i}=\bm{g}|x_{u,i})}{\partial
g_{s}^{2}}\cdot\big{(}\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g})\big{)}\Big{\\}}\pi(\bm{g})d\bm{g}\right]$
$\displaystyle+\frac{4M_{p}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\mathcal{R}(\mathcal{F})+\frac{5M_{p}M_{|\delta-\hat{\delta}|}(M_{K})^{q}}{\prod_{s=1}^{q}h_{s}}\sqrt{\frac{2}{|\mathcal{D}|}\log(\frac{4}{\eta})}+o(\sum_{s=1}^{q}h_{s}^{2}).$

###### Proof of Theorem 8.

This conclusion can be derived similarly from the proof of Theorem 5. ∎

##  Appendix D Pseudo-Codes for Propensity Learning, N-IPS, N-DR-JL and
N-MRDR-JL

Input: the observation matrix $\mathbf{O}$ and the representation space
$\mathcal{G}$.

1 Using previous methods such as logistic regression to train a model to
estimate $\mathbb{P}(o=1\mid\bm{x})$;

2 while  _stopping criteria is not satisfied_ do

3 Sample a batch of user-item pairs $\left\\{(u_{j},i_{j})\right\\}_{j=1}^{J}$
with $o_{u,i}=1$ to generate samples
$\left\\{(\bm{x}_{u_{j},i_{j}},\bm{g}_{u_{j},i_{j}})\right\\}$ with positive
label $(L=1)$;

4 Uniformly sample a batch of treatments
$\\{\bm{g}_{u_{k},i_{k}}^{\prime}\\}_{k=1}^{K}\subset\mathcal{G}$ to generate
samples
$\left\\{(\bm{x}_{u_{j},i_{j}},\bm{g}_{u_{k},i_{k}}^{\prime})\right\\}$ with
negative label $(L=0)$;

5 Using gradient descent to train a logistic regression model that estimates
$\mathbb{P}(L=1\mid\bm{x},\bm{g})$ using the positive samples and negative
samples;

6

7 end while

Output: the propensity model that estimates
$\mathbb{P}(L=1\mid\bm{x},\bm{g})$.

Algorithm 1 The Proposed Propensity Learning Algorithm

To learn the propensity model $p_{u,i}(\bm{g})$, we have the following holds
that

$\frac{1}{p_{u,i}(\bm{g})}=\frac{c}{\mathbb{P}(o=1\mid\bm{x})}\cdot\frac{\mathbb{P}(L=1)}{\mathbb{P}(L=0)}\cdot\frac{\mathbb{P}(L=0\mid\bm{x},\bm{g})}{\mathbb{P}(L=1\mid\bm{x},\bm{g})}\propto\frac{1}{\mathbb{P}(o=1\mid\bm{x})}\cdot\frac{\mathbb{P}(L=0\mid\bm{x},\bm{g})}{\mathbb{P}(L=1\mid\bm{x},\bm{g})}.$

The constant $c$ and ${\mathbb{P}(L=1)}/{\mathbb{P}(L=0)}$ can be ignored and
$\mathbb{P}(o=1\mid\bm{x})$ can be estimated by using the existing methods
such as naive Bayes or logistic regression with or without a few unbiased
ratings, respectively. Therefore, we need to train the model that estimates
$\mathbb{P}(L=1\mid\bm{x},\bm{g})$, as shown in Algorithm 1. In addition, the
learning algorithms for N-IPS, N-DR-JL and N-MRDR-JL are shown in Algorithms
2-4. Specifically, for the N-MRDR-JL learning algorithm, the imputation model
parameters are optimized by minimizing the following loss

$\displaystyle\mathcal{L}_{e}^{\mathrm{N-MRDR}}(\hat{\bf
R})=\int|\mathcal{D}|^{-1}\sum_{(u,i)\in\mathcal{D}}{\frac{\mathbb{I}(o_{u,i}=1)\cdot(1-p_{u,i}(\bm{g}))\cdot
K\left((\bm{g}_{u,i}-\bm{g})/h\right)\cdot(\delta_{u,i}(\bm{g})-\hat{\delta}_{u,i}(\bm{g}))^{2}}{h\cdot
p^{2}_{u,i}(\bm{g})}}\pi(\bm{g})d\bm{g}.$

Compared to the $\mathcal{L}_{e}^{\mathrm{N-DR}}(\hat{\bf R})$, the
$\mathcal{L}_{e}^{\mathrm{N-MRDR}}(\hat{\bf R})$ loss has variance reduction
property.

Input: the observed ratings $\mathbf{Y}^{o}$, the representation space
$\mathcal{G}$, the pre-specified $\pi({\bm{g}})$, the pre-specified kernel
function $K(\cdot)$ and the propensity model.

1 while  _stopping criteria is not satisfied_ do

2 Sample a batch of user-item pairs $\left\\{(u_{j},i_{j})\right\\}_{j=1}^{J}$
from $\mathcal{O}$;

3 Calculate $\hat{p}_{u_{j},i_{j}}(\bm{g})$ using the propensity model;

4 Update $\theta$ by descending along the gradient
$\nabla_{\theta}\mathcal{L}_{\mathrm{IPS}}^{\mathrm{N}}(\hat{\bf R})$;

5

6 end while

Output: the debiased prediction model $f_{\theta}(x)$.

Algorithm 2 The Proposed N-IPS Learning Algorithm

Input: the observed ratings $\mathbf{Y}^{o}$, the observation matrix
$\mathbf{O}$, the representation space $\mathcal{G}$, the pre-specified
$\pi({\bm{g}})$, the pre-specified kernel function $K(\cdot)$ and the
propensity model.

while  _stopping criteria is not satisfied_ do

for  _number of steps for training the imputation model_ do

Sample a batch of user-item pairs $\left\\{(u_{j},i_{j})\right\\}_{j=1}^{J}$
from $\mathcal{O}$;

Calculate $\hat{p}_{u_{j},i_{j}}(\bm{g})$ using the propensity model;

Update $\phi_{\bm{g}}$ by descending along the gradient
$\nabla_{\phi_{\bm{g}}}\mathcal{L}_{e}^{\mathrm{N-DR}}(\hat{\bf R})$;

end for

for  _number of steps for training the prediction model_ do

Sample a batch of user-item pairs $\left\\{(u_{k},i_{k})\right\\}_{k=1}^{K}$
from $\mathcal{D}$;

Calculate $\hat{p}_{u_{k},i_{k}}(\bm{g})$ using the propensity model for user-
item pair with $o_{u_{k},i_{k}}=1$;

Update $\theta$ by descending along the gradient
$\nabla_{\theta}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})$;

end for

end while

Output: the debiased prediction model $f_{\theta}(x)$.

Algorithm 3 The Proposed N-DR-JL Learning Algorithm

Input: the observed ratings $\mathbf{Y}^{o}$, the observation matrix
$\mathbf{O}$, the representation space $\mathcal{G}$, the pre-specified
$\pi({\bm{g}})$, the pre-specified kernel function $K(\cdot)$ and the
propensity model.

while  _stopping criteria is not satisfied_ do

for  _number of steps for training the imputation model_ do

Sample a batch of user-item pairs $\left\\{(u_{j},i_{j})\right\\}_{j=1}^{J}$
from $\mathcal{O}$;

Calculate $\hat{p}_{u_{j},i_{j}}(\bm{g})$ using the propensity model;

Update $\phi_{\bm{g}}$ by descending along the gradient
$\nabla_{\phi_{\bm{g}}}\mathcal{L}_{e}^{\mathrm{N-MRDR}}(\hat{\bf R})$;

end for

for  _number of steps for training the prediction model_ do

Sample a batch of user-item pairs $\left\\{(u_{k},i_{k})\right\\}_{k=1}^{K}$
from $\mathcal{D}$;

Calculate $\hat{p}_{u_{k},i_{k}}(\bm{g})$ using the propensity model for user-
item pair with $o_{u_{k},i_{k}}=1$;

Update $\theta$ by descending along the gradient
$\nabla_{\theta}\mathcal{L}_{\mathrm{DR}}^{\mathrm{N}}(\hat{\bf R})$;

end for

end while

Output: the debiased prediction model $f_{\theta}(x)$.

Algorithm 4 The Proposed N-MRDR-JL Learning Algorithm

##  Appendix E Semi-synthetic Experiment Details

Following the previous studies [Schnabel et al., 2016, Wang et al., 2019, Guo
et al., 2021], we set propensity
$p_{u,i}=p\alpha^{\max\left(0,4-r_{u,i}\right)}$ for each user-item pair to