# 3D Shape Generation and Completion through Point-Voxel Diffusion

Linqi Zhou
Stanford University  Yilun Du
MIT  Jiajun Wu
Stanford University

###### Abstract

We propose a novel approach for probabilistic generative modeling of 3D
shapes. Unlike most existing models that learn to deterministically translate
a latent vector to a shape, our model, Point-Voxel Diffusion (PVD), is a
unified, probabilistic formulation for unconditional shape generation and
conditional, multi-modal shape completion. PVD marries denoising diffusion
models with the hybrid, point-voxel representation of 3D shapes. It can be
viewed as a series of denoising steps, reversing the diffusion process from
observed point cloud data to Gaussian noise, and is trained by optimizing a
variational lower bound to the (conditional) likelihood function. Experiments
demonstrate that PVD is capable of synthesizing high-fidelity shapes,
completing partial point clouds, and generating multiple completion results
from single-view depth scans of real objects. ††footnotetext: Project page at
https://alexzhou907.github.io/pvd

Figure 1: The proposed Point-Voxel Diffusion (PVD) is a new framework for
generative modeling of 3D shapes. Left: tables, cars, and planes generated by
our PVD. It learns to sample from a Gaussian prior and to progressively remove
noise to obtain sharp shapes. Right: two possible shapes completed from a real
RGB-D image, each visualized in input and canonical views.

##  1 Introduction

Generative modeling of 3D shapes has extensive applications across vision,
graphics, and robotics. To perform well in these downstream applications, a
good 3D generative models should be _faithful_ and _probabilistic_. A faithful
model generates shapes that are realistic to humans and, in cases where
conditional inputs such as depth maps are available, respects such partial
observations. A probabilistic model captures the under-determined, multi-modal
nature of the generation and completion problem: it may sample and produce
diverse shapes from scratch or from partial observations. As shown in Figure
3D Shape Generation and Completion through Point-Voxel Diffusion, when only
the back of a chair is visible, good generative models should be able to
produce multiple possible completed chairs, including those with arms and
those without.

Existing shape generation models can be roughly divided into two categories.
The first operates on 3D voxels [43, 14, 45, 2], a natural extension of 2D
pixels. While being straightforward to use, voxels demand prohibitively large
memory when scaled to high dimensions, and are thus unlikely to produce
results with high fidelity. The second class of models studies point cloud
generation [1, 11, 48, 46, 16, 17] and has produced promising results. While
being more faithful, these approaches typically view point cloud generation as
a point generation process conditioned on shape encoding, which is obtained by
deterministic encoders. When performing shape completion, these approaches are
therefore unable to capture the multi-modal nature of the completion problem.

Recently, a new class of generative models, named probabilistic diffusion
models, have achieved impressive performance on 2D image generation [35, 13,
36]. These approaches learn a probabilistic model over a denoising process.
Diffusion is supervised to gradually denoise a Gaussian noise to a target
output, such as an image. Methods along this line, such as DDPM [13], are
inherently probabilistic and produce highly realistic 2D images.

Extending diffusion models to 3D is, however, technically highly nontrivial: a
direct application of diffusion models on either voxel and point
representation results in poor generation quality. This is because, first,
pure voxels are binary and therefore not suitable for the probabilistic nature
of diffusion models; second, point clouds demand permutation-invariance, which
imposes infeasible constraints on the model. Experiments in Section 4.1 also
verifies that a straightforward extension does not lead to reasonable results.

We propose Point-Voxel Diffusion (PVD), a probabilistic and flexible shape
generation model that tackles the above challenges by marrying denoising
diffusion models with the hybrid, point-voxel representation of 3D shapes
[24]. A point-voxel representation builds structured locality into point cloud
processing; integrated with denoising diffusion models, PVD suggests a novel,
probabilistic way to generate high-quality shapes by denoising a Gaussian
noise and to produce multiple completion results from a partial observation,
as shown in Figure 3D Shape Generation and Completion through Point-Voxel
Diffusion.

A unique strength of PVD is that it is a unified, probabilistic formulation
for unconditional shape generation and conditional, multi-modal shape
completion. While multi-modal shape completion is a highly desirable feature
in applications such as digital design or robotics, past works on shape
generation primarily use deterministic shape encoders and decoders to output a
single possible completion in voxels or a point cloud. In contrast, PVD can
perform both unconditional shape generation and conditional shape completion
in an integrated framework, requiring only minimal modifications to the
training objective. It is thus capable of sampling multiple completion results
depending on diffusion initialization.

Experiments demonstrate that PVD is capable of synthesizing high-fidelity
shapes, outperforming multiple state-of-the-art methods. PVD also delivers
high-quality results on multi-modal shape completion from partial observations
such as a partial point cloud or a depth map. In particular, we show that PVD
does well on multi-modal completion on multiple synthetic and real datasets,
including ShapeNet [4], PartNet [27], and single-view depth scans of real
objects in the Redwood dataset [5].

##  2 Related Works

#### Point cloud generative models.

Many prior works have explored point cloud generation in terms of auto-
encoding [1, 11, 47], single-view reconstruction [12, 10, 18, 17], and
adversarial generation [34, 48, 40]. Many of them rely on directly optimizing
heuristic loss functions such as Chamfer Distance (CD) and Earth Mover’s
Distance (EMD), which are also used to evaluate generative quality.

Some recent works take a different approach, viewing the 3D point clouds in
light of probabilistic distributions. For example, Sun  _et al_. [37] view the
point clouds from a probabilistic perspective and introduce autoregressive
generation, but doing so requires ordering of the point clouds. GAN-based
models and flow-based models [21, 1, 46, 16, 17] also adopt a probabilistic
view but separate shape-level distribution from point-level distribution.
Among these models, PointFlow applies normalizing flow [29] to 3D point
clouds, and Discrete PointFlow follows up using discrete normalizing flow with
affine coupling layers [8]. Shape Gradient Fields [3], unlike flow-based
works, directly learn a gradient field that samples point clouds using
Langevin dynamics. Our model is different from these models in that we do not
distinguish point and shape distributions, and that we directly generate
entire shapes starting from random noise.

#### Point-voxel representation.

3D shapes were conventionally rasterized into voxel grids and processed using
3D convolution [6, 42]. Due to the correspondence between voxels and 2D
pixels, many works have explored voxel-based classification and segmentation
using volumetric convolution [26, 31, 19, 39, 42, 7]. Voxel-based generative
models have similarly proven successful [43, 14, 45]. However, voxel grids are
memory-intensive and they grow cubically with increase in dimension, so they
cannot be scaled to a high resolution.

Point clouds, on the other hand, are detailed samples from smooth surfaces and
do not suffer from the grid effect of usually low-resolution voxels and do not
require as much memory for processing. Researchers have explored point cloud
classification and segmentation [30, 30, 41] and most assume point cloud
processing networks are permutation-invariant. Permutation-invariance is a
strong condition to be imposed on the architecture and we empirically find
that direct extension of 2D methods to either permutation-invariant point
clouds or voxels do not work well. We therefore explore a separate point-voxel
representation [22, 33], and our work is most related to point-voxel CNN [24],
which proposes to voxelize the point clouds for 3D convolution. We use it as
the backbone of our generative model due to its exploitation of the strong
spacial correlation inherent in point cloud data.

#### Energy-based models and denoising diffusion models.

Energy-based models (EBMs) and denoising diffusion models are two classes of
generative models that formulate generation as an iterative refinement
procedure. Energy based models [20, 9, 28] learn an energy landscape over
input data, where local minima correspond to high-fidelity samples, which are
obtained by Langevin dynamics [9, 28]. In contrast, denoising diffusion models
[35, 13, 36] learn a probabilistic model over a denoising process on inputs.
Diffusion is supervised to gradually denoise a Gaussian noise to a target
output. This form of supervision can be seen as supervision of the gradient of
a log probability distribution [36] as in score matching EBM [15, 38]. Our
work builds on these related existing approaches and we explore the 3D domain,
which is challenging and fundamentally different from 2D images. A concurrent
work on point cloud diffusion model [25] views point cloud generation as a
conditional generation problem and uses an additional encoder for shape
latents. Ours, however, adopts an unconditional approach, ridding the need for
additional shape encoders, and uses a different hybrid, point-voxel
representation for processing shapes. In addition to generating high-quality
3D shapes, we also show that our model can be modified with no architectural
change to perform on conditional generation tasks such as shape completion. We
also demonstrate its effectiveness on real-world scans.

Figure 2: Visualization of the diffusion and generative process. To generate,
Gaussian noise is sampled from $p({\bf x}_{T})$ and noise is progressively
removed by $p_{\theta}({\bf x}_{t}|{\bf x}_{t+1})$. Symmetrically, the
diffusion process gradually adds noise by $q({\bf x}_{t+1}|{\bf x}_{t})$. We
utilize a closed-form expression for each $q({\bf x}_{t+1}|{\bf x}_{t})$,
allowing $p_{\theta}({\bf x}_{t}|{\bf x}_{t+1})$ to be learned by simply
matching the posterior $q({\bf x}_{t}|{\bf x}_{t+1},{\bf x}_{0})$ of the
corresponding forward transition probability.

##  3 Point-Voxel Diffusion

In this section we introduce Point-Voxel Diffusion (PVD), a denoising
diffusion probabilistic model for 3D point clouds. We start by describing our
formulation, followed by the training objective for shape generation, and end
with the modified objective we proposed for shape completion from partial
observation. For all our discussions below, we assume each of our data points
are a set of $N$ points with $xyz$-coordinates and is denoted as ${\bf
x}\in\mathbb{R}^{N\times 3}$. Our model is parameterized as a single point-
voxel CNN [24].

###  3.1 Formulation

The denoising diffusion probabilistic model is a generative model where
generation is modeled as a denoising process. Starting from Gaussian noise,
denoising is performed until a sharp shape is formed. In particular, the
denoising process produces a series of shape variables with decreasing levels
of noise, denoted as ${\bf x}_{T},{\bf x}_{T-1},...,{\bf x}_{0}$, where ${\bf
x}_{T}$ is sampled from a Gaussian prior and ${\bf x}_{0}$ is the final
output.

To learn our generative model, we define a ground truth diffusion distribution
$q({\bf x}_{0:T})$ (defined by gradually adding Gaussian noise to the ground
truth shape), and learn a diffusion model $p_{\theta}({\bf x}_{0:T})$, which
aims to invert the noise corruption process. We factor both probability
distributions into products of Markov transition probabilities:

$\displaystyle q({\bf x}_{0:T})$ $\displaystyle=q({\bf
x}_{0})\prod_{t=1}^{T}q({\bf x}_{t}|{\bf x}_{t-1}),$ (1) $\displaystyle
p_{\theta}({\bf x}_{0:T})$ $\displaystyle=p({\bf
x}_{T})\prod_{t=1}^{T}p_{\theta}({\bf x}_{t-1}|{\bf x}_{t}),$

where $q({\bf x}_{0})$ is the data distribution and $p({\bf x}_{T})$ is a
standard Gaussian prior. Here, $q({\bf x}_{t}|{\bf x}_{t-1})$ is named the
forward process, diffusing data into noise; accordingly, $q({\bf x}_{t-1}|{\bf
x}_{t})$ is named the reverse process. $p_{\theta}({\bf x}_{t-1}|{\bf x}_{t})$
is named the generative process, which we learn, that generates realistic
samples by approximates the reverse process. To enable closed-form evaluation,
the transition probabilities are also parameterized as Gaussian distributions.
We illustrate the processes in Figure 2. Given a pre-determined increasing
sequence of Gaussian noise values $\beta_{1},...,\beta_{T}$,111We leave
derivation and implementation details to Appendix. each transition probability
can be defined as

$\displaystyle q({\bf x}_{t}|{\bf x}_{t-1})$
$\displaystyle:={\mathcal{N}}(\sqrt{1-\beta_{t}}{\bf x}_{t-1},\beta_{t}{\bf
I}),$ (2) $\displaystyle p_{\theta}({\bf x}_{t-1}|{\bf x}_{t})$
$\displaystyle:={\mathcal{N}}(\mu_{\theta}({\bf x}_{t},t),\sigma_{t}^{2}{\bf
I}).$

, where $\mu_{\theta}({\bf x}_{t},t)$ represents the predicted shape from our
generative model at timestep $t-1$. Empirically, we found that setting
$\sigma_{t}^{2}=\beta_{t}$ works well. Intuitively, the forward process can be
seen as gradually injecting more random noise to the data, with the generative
process learning to progressively remove noise to obtain realistic samples by
mimicking the reverse process.

#### Training objective.

To learn the marginal likelihood $p_{\theta}({\bf x})$, we maximize a
variational lower bound of log data likelihood that involves all of ${\bf
x}_{0},...,{\bf x}_{T}$:

${\rm E}_{q({\bf x}_{0})}[\log p_{\theta}({\bf x}_{0})]\geq{\rm E}_{q({\bf
x}_{0:T})}\Big{[}\log\frac{p_{\theta}({\bf x}_{0:T})}{q({\bf x}_{1:T}|{\bf
x}_{0})}\Big{]}.$ (3)

In the above objective, the forward process $q({\bf x}_{t}|{\bf x}_{t-1})$ is
fixed and $p({\bf x}_{T})$ is defined as a Gaussian prior, so they do not
affect the learning of $\theta$. Therefore, the final objective can be reduced
to maximum likelihood given the complete data likelihood with joint posterior
$q({\bf x}_{1:T}|{\bf x}_{0})$:

$\max_{\theta}{\rm E}_{{\bf x}_{0}\sim q({\bf x}_{0}),{\bf x}_{1:T}\sim q({\bf
x}_{1:T}|{\bf x}_{0})}\left[\sum_{t=1}^{T}\log p_{\theta}({\bf x}_{t-1}|{\bf
x}_{t})\right].$ (4)

Joint posterior $q({\bf x}_{1:T}|{\bf x}_{0})$ can be factorized into
$\prod_{t=1}^{T}{q({\bf x}_{t-1}|{\bf x}_{t},{\bf x}_{0})}$. Each factored
ground-truth posterior is denoted as $q({\bf x}_{t-1}|{\bf x}_{t},{\bf
x}_{0})$ and is analytically tractable. It can be shown that it is also
parameterized by Gaussian distributions:

$\displaystyle q({\bf x}_{t-1}|{\bf x}_{t},{\bf x}_{0})=$ (5)
$\displaystyle{\mathcal{N}}\left(\frac{\sqrt{\tilde{\alpha}_{t-1}}\beta_{t}}{1-\tilde{\alpha}_{t}}{\bf
x}_{0}+\frac{\sqrt{\alpha_{t}}(1-\tilde{\alpha}_{t-1})}{1-\tilde{\alpha}_{t}}{\bf
x}_{t},\frac{(1-\tilde{\alpha}_{t-1})}{1-\tilde{\alpha}_{t}}\beta_{t}{\bf
I}\right).$

where $\alpha_{t}=1-\beta_{t}$ and
$\tilde{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$. This property allows each
timestep to learn independently, _i.e_., each $p_{\theta}({\bf x}_{t-1}|{\bf
x}_{t})$ only needs to match $q({\bf x}_{t-1}|{\bf x}_{t},{\bf x}_{0})$.

Since both $p_{\theta}({\bf x}_{t-1}|{\bf x}_{t})$ and $q({\bf x}_{t-1}|{\bf
x}_{t},{\bf x}_{0})$ are Gaussian, we can reparameterize the model to output
noise and the final loss can be reduced to an $\mathcal{L}_{2}$ loss between
the model output $\bm{\epsilon}_{\theta}({\bf x}_{t},t)$ and noise
$\bm{\epsilon}$:

$\left\lVert\bm{\epsilon}-\bm{\epsilon}_{\theta}({\bf
x}_{t},t)\right\rVert^{2},\;\;\bm{\epsilon}\sim{\mathcal{N}}(0,{\bf I}),$ (6)

Intuitively, the model seeks to predict the noise vector necessary to
decorrupt the 3D shape.

Point clouds can then be generated by progressively sampling from
$p_{\theta}({\bf x}_{t-1}|{\bf x}_{t})$ as $t=T,...,1$ using the following
equation:

${\bf x}_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}\left({\bf
x}_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\tilde{\alpha}_{t}}}\bm{\epsilon}_{\theta}({\bf
x}_{t},t)\right)+\sqrt{\beta_{t}}\mathbf{z},$ (7)

where $\mathbf{z}\sim{\mathcal{N}}(0,{\bf I})$, corresponding to the gradual
denoising of a shape from noise.

###  3.2 Shape Completion

Our objective can be simply modified to learn a conditional generative model
given partial shapes, which we introduce in this section.

Denote a point cloud sample as ${\bf x}_{0}=({\bf z}_{0},\tilde{{\bf
x}}_{0})$, where ${\bf z}_{0}\in\mathbb{R}^{M\times 3}$ is the fixed partial
shape, and any intermediate shapes as free points ${\bf x}_{t}=({\bf
z}_{0},\tilde{{\bf x}}_{t})$. We can then define a conditional forward
process, where the partial shape is fixed at ${\bf z}_{0}$ for all time. Our
conditional forward and generative processes, as well as each transition
probability, can then be parametrized as

$\displaystyle q(\tilde{{\bf x}}_{t}|\tilde{{\bf x}}_{t-1},{\bf z}_{0})$
$\displaystyle:={\mathcal{N}}(\sqrt{1-\beta_{t}}\tilde{{\bf
x}}_{t-1},\beta_{t}{\bf I}),$ (8) $\displaystyle p_{\theta}(\tilde{{\bf
x}}_{t-1}|\tilde{{\bf x}}_{t},{\bf z}_{0})$
$\displaystyle:={\mathcal{N}}(\mu_{\theta}({\bf x}_{t},{\bf
z}_{0},t),\sigma_{t}^{2}{\bf I}).$

Note that the above equations now give the forward/generative transition
probabilities for the free points $\tilde{{\bf x}}_{t}$, while ${\bf z}_{0}$
stays unchanged for all timesteps. Intuitively, this process is the same as
unconditional generation, while we hold the partial shape ${\bf z}_{0}$ fixed
and diffuse only the missing parts.

The modified training objective also maximizes the likelihood conditioned on
partial shapes ${\bf z}_{0}$:

${\rm E}_{(\tilde{{\bf x}}_{0},{\bf z}_{0})\sim q(x_{0}),{\bf x}_{1:T}\sim
q({\bf x}_{1:T}|\tilde{{\bf x}}_{0},{\bf z}_{0})}\left[\sum_{t=1}^{T}\log
p_{\theta}(\tilde{{\bf x}}_{t-1}|\tilde{{\bf x}}_{t},{\bf z}_{0})\right],$ (9)

where each posterior $q(\tilde{{\bf x}}_{t-1}|\tilde{{\bf x}}_{t},\tilde{{\bf
x}}_{0},{\bf z}_{0})$ is known and its derivation is similar to the
unconditional generative model. Using the same reasoning as before, we can
arrive at a similar $\mathcal{L}_{2}$ loss:

$\mathcal{L}_{t}=\left\lVert\bm{\epsilon}-\bm{\epsilon}_{\theta}(\tilde{{\bf
x}}_{t},{\bf z}_{0},t)\right\rVert^{2},$ (10)

where $\bm{\epsilon}\sim{\mathcal{N}}(0,{\bf I})$. Additionally, since the
partial shape is always fixed during both forward and generative processes, we
can mask away the subset of model output that affects ${\bf z}_{0}$ and
minimize $\mathcal{L}_{2}$ distance between $\tilde{\bm{\epsilon}}(\tilde{{\bf
x}}_{t},{\bf z}_{0},t)$ and random noise, which only affects $\tilde{{\bf
x}}_{t}$. In practice, we input ${\bf z}_{0}$ and ${\bf x}_{t}$ into the model
and obtain ${\bf x}_{t-1}$, where only the subset $\tilde{{\bf x}}_{t-1}$ is
used for $\mathcal{L}_{2}$ loss. In shape completion, $\tilde{{\bf x}}_{t-1}$
is concatenated with ${\bf z}_{0}$ to be the input into the model again. This
allows the exact same training architecture to do both generation and shape
completion by simply changing the training objective.

|  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
l-GAN | PointFlow | SoftFlow | DPF-Net | Shape-GF | Vox-Diff | PVD (ours)

Figure 3: Results on unconditional shape generation with 2,048 points. The
l-GAN results are from the EMD variant.

##  4 Experiments

We demonstrate here that our model outperforms previous point generative
models in Section 4.1, is capable of completing partial shapes sampled from
single views in Section 4.2, and can generate diverse shapes given partial
shape constraints in Section 4.3. Architecture and hyper-parameter details are
provided in Appendix.

###  4.1 Shape Generation

| Airplane | Chair | Car
---|---|---|---
| CD | EMD | CD | EMD | CD | EMD
r-GAN [1] | 98.40 | 96.79 | 83.69 | 99.70 | 94.46 | 99.01
l-GAN (CD) [1] | 87.30 | 93.95 | 68.58 | 83.84 | 66.49 | 88.78
l-GAN (EMD) [1] | 89.49 | 76.91 | 71.90 | 64.65 | 71.16 | 66.19
PointFLow [46] | 75.68 | 70.74 | 62.84 | 60.57 | 58.10 | 56.25
SoftFlow [16] | 76.05 | 65.80 | 59.21 | 60.05 | 64.77 | 60.09
DPF-Net [17] | 75.18 | 65.55 | 62.00 | 58.53 | 62.35 | 54.48
Shape-GF [3] | 80.00 | 76.17 | 68.96 | 65.48 | 63.20 | 56.53
PVD (ours) | 73.82 | 64.81 | 56.26 | 53.32 | 54.55 | 53.83
Table 1: Generation results on Airplane, Chair, Car compared with baselines
using 1-NN as the metric. Both CD and EMD as the distance measure are
calculated. Lower scores indicate better quality and diversity.

#### Data.

We choose ShapeNet [4] Airplane, Chair, and Car to be our main datasets for
generation, following most previous works [46, 17, 3, 16]. We use the provided
datasets in [46], which contain 15,000 sampled points for each shape. We
sample 2,048 points for training and testing, respectively, and process our
data following procedures provided in PointFlow [46].

#### Evaluation metrics.

Previous works such as [46, 17, 3, 16] have used Chamfer Distance (CD) and
Earth Mover’s Distance (EMD) as their distance metrics in calculating Jensen-
Shannon Divergence (JSD), Coverage (COV), Minimum Matching Distance (MMD), and
1-Nearest Neighbor (1-NN), which are four main metrics to measure generative
quality. However, as discussed by [46], JSD, COV, and MMD each has limitations
and does not necessarily indicate better quality. Some generation results
achieve even better scores than ground-truth datasets. 1-NN is robust and
correlates with generation quality, as supported by [46], which also proposes
1-NN as the better metric. Therefore, we use 1-NN directly for evaluating
generation quality and we provide comparison of remaining metrics in Appendix.
As we also discover that EMD score can vary widely depending on its
implementation, We evaluate all baselines using our implementation of the
metrics.

#### Baselines and results.

We quantitatively compare our results with r-GAN [1], l-GAN [1], PointFlow
[46], DPF-Net [17], SoftFlow [16], and Shape-GF [3] on generating 2048 points.
In evaluating the baselines, we follow the same data processing and evaluation
procedure as PointFlow, and follow the provided baseline implementations to
evaluate their models. Our comparisons are shown in Table 1. Our model
noticeably achieves the best generation quality.

We also investigated pure voxel and point representations for shape
generation. We noticed that simply extending diffusion models to pure point
representation using conventional permutation-invariant architectures such as
PointNet++ [32] fails to generate any visible shapes. Extending diffusion
models to pure voxel representation generates noisy results due to the binary
nature of voxels which is different from our Gaussian assumption. We visually
compare with baselines including a voxel diffusion model (Vox-Diff) in Figure
3. For Vox-Diff, 2048 points are sampled from voxel surfaces. We provide
additional quantitative comparison in Appendix.

|  |  |  |  |  |
---|---|---|---|---|---|---
|  |  |  |  |  |
|  |  |  |  |  |
Depths | Ground-truths | Partial | PointFlow | SoftFlow | DPF-Net | PVD (ours)

Figure 4: Our shape completion visualization (right) compared to baseline
models (left). From left to right: depth images, ground-truth shapes, partial
shapes sampled from depth images, completion from baselines, and our results.

###  4.2 Shape Completion

In various graphics applications users usually do not have access to all
viewpoints of an object. A shape often needs to be completed knowing a partial
shape from a single depth map. Therefore, the ability to complete partial
shapes become practically useful. In this section, we use the same model
architecture (see Appendix) from Section 4.1 and test our shape completion
models.

#### Data.

For shape completion, we use the benchmark provided by GenRe [49], which
contains renderings of each shape in ShapeNet from 20 random views. We sample
200 points as our partial point clouds obtained from the provided depth
images, and we evaluate shape completion on all 20 partial shapes per ground-
truth sample.

#### Metrics.

For shape completion, as the ground-truth data are involved, Chamfer Distance
and Earth Mover’s Distance suffice to evaluate the reconstruction results.

#### Baselines.

Since our approach is probabilistic, we selected major distribution-fitting
models such as PointFlow [46], DPF-Net [17], and SoftFlow [16] for comparison.
We directly evaluate pre-trained models, if provided, otherwise we re-train
them using baselines’ provided implementation on our benchmark. We also
compared with Shape-GF as its encoder can similarly receive an arbitrary
number of points. However, it is experimentally found that the model is
sensitive to the input partial shapes and completion is not realistic after
Langevin sampling. Therefore, we leave them out of the comparison.

Category | Model | CD | EMD
---|---|---|---
Airplane | SoftFlow [16] | 0.4042 | 1.198
PointFlow [46] | 0.4030 | 1.180
DPF-Net [17] | 0.5279 | 1.105
PVD (ours) | 0.4415 | 1.030
Chair | SoftFlow [16] | 2.786 | 3.295
PointFlow [46] | 2.707 | 3.649
DPF-Net [17] | 2.763 | 3.320
PVD (ours) | 3.211 | 2.939
Car | SoftFlow [16] | 1.850 | 2.789
PointFlow [46] | 1.803 | 2.851
DPF-Net [17] | 1.396 | 2.318
PVD (ours) | 1.774 | 2.146
Table 2: Quantitative comparison against baselines. CD is multiplied by
$10^{3}$ and EMD is multiplied by $10^{2}$.

GT | PointFlow | SoftFlow | DPF-Net | PVD (ours)
---|---|---|---|---
CD | 8.633 | 1.501 | 1.197 | 1.921
EMD | 1.124 | 1.856 | 1.109 | 0.919

Figure 5: Typical case when CD is higher than baseline models. Column 1 shows
input depth image and ground-truth point clouds. The next columns show
completion from the input viewpoint (top) and from the canonical viewpoint
(bottom). CD is multiplied by $10^{3}$ and EMD is multiplied by $10^{2}$
scores.

#### Results.

Quantitative results are presented in Table 2 and a visual comparison is shown
in Figure 4. From Table 2, we observe that our model achieves best on EMD
scores while worse on CD compared to some baselines. First, we note EMD is a
better metric for measuring completion quality because by solving the linear
assignment problem it forces model outputs to have the same density as the
ground-truths [23] and it is known that CD is blind to visual inferiority [1].
Our better EMD score is more indicative of higher visual quality.

|  |  |  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |
Partial |  KNN-latent |  cGAN |  PVD (Ours) |  Partial |  KNN-latent |  cGAN |  PVD (Ours) |  Partial |  KNN-latent |  cGAN |  PVD (Ours)
Figure 6: Multi-modal completion visualization on PartNet. Each column
presents five completion modes from a model.

Next, we investigate the reason our CD is inferior to our baselines. We
discover that a typical case when our CD is higher is as shown in Figure 5,
where from the input view the ground-truth shape is largely unknown. The
baseline models tend to output a mean shape when encountered with such an
unconventional angle. Naturally, mean shapes are more frequently closer to the
ground-truths than other shapes, as exemplified by the figure. However, with
each noise initialization, our model seeks a possible completion that matches
well with the partial shape and may be further away from the ground-truth than
the mean shape. In the case shown, our completion is a van instead of a sedan
but is equally realistic.

Our model also enables controlled completion given multiple partial shapes,
and we leave details to Appendix.

|  TMD $\times 10^{2}$ |  MMD $\times 10^{3}$
---|---|---
| Chair | Table | Lamp | Avg. | Chair | Table | Lamp | Avg.
KNN-latent [44] | 0.96 | 1.37 | 1.95 | 1.43 | 1.42 | 1.42 | 1.88 | 1.57
cGAN [44] | 1.75 | 1.99 | 1.94 | 1.89 | 1.61 | 1.56 | 2.13 | 1.77
PVD (ours) | 1.91 | 1.70 | 5.92 | 3.18 | 1.27 | 1.03 | 1.98 | 1.43
Table 3: Quantitative comparison for multi-modal completion on PartNet. TMD
(higher the better) measures completion diversity, and MMD (lower the better)
measures completion quality. Chamfer Distance (CD) is used as the distance
measure.

###  4.3 Multi-Modal Completion

Our baselines for shape completion adopt an encoder-decoder structure that
takes in a partial shape and outputs a single completion. While some offer
impressive results, their completion ability is deterministic, much different
from humans who can often imagine different completion possibilities given
single views. Our PVD, however, adopts a probabilistic approach to shape
completion, where each noise initialization can result in a different
completion.

#### Data.

We follow experiment setups from cGAN [44] and train our model on Chair,
Table, and Lamp from PartNet [27]. 1024 points are given and 2048 points are
generated as completion. In addition, we show diversity of our completion on
ShapeNet. Different from PartNet, ShapeNet’s partial shapes are sampled from
custom-rendered depth images.

#### Metrics.

We follow cGAN [44] and use Total Mutual Difference (TMD) to measure diversity
and Minimal Matching Distance (MMD) to measure quality with Chamfer Distance
(CD) as distance measure. Since our model is different from cGAN and only
completes the 1024 free points, TMD is calculated only on the free points for
our model and on a subsampled set of 1024 points for our baselines. MMD is
calculated using the completed 2048 points and re-sampled 2048 ground-truth
points.

#### Results on PartNet.

Our model is compared with cGAN [44] and KNN-latent [44]. Results are shown in
Table 3 and visual comparisons are shown in Figure 6. Our model outperforms
both baselines in terms of average diversity and quality.

Figure 7: Multi-modal completion results on ShapeNet. Left: ground-truth
bottom view depth image of a chair. Right: six different possible shape
completion results. Top: completion from the depth image viewpoint. Bottom:
completion from the canonical viewpoint.

|  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |
RGB-D | GT | Completion Results | RGB-D | GT | Completion Results

Figure 8: Application of our model on scans from the Redwood 3DScans dataset.
PVD takes partial point clouds induced from the depth maps, not the RGB image
as input. Left: from a more complete view, the model outputs stable, similar
completions. Right: from an uncertain viewpoint, the model outputs multiple
completions with a larger variation.

#### Results on ShapeNet.

The shape completion model trained in Section 4.2 is directly used to
demonstrate completion diversity on ShapeNet, as shown in Figure 7. We choose
a bottom view of a chair and show that, in the top row, all of our completion
results match well with the constrained viewpoint, and in the bottom row, our
completion results are noticeably diverse from the canonical viewpoint.

#### Results on real scans.

We further investigate how our model pre-trained on ShapeNet can perform on
scans of real objects. We use the Redwood 3DScans dataset [5] and test our
model on partial shapes of chairs and tables, sampled from its depth images.
Since the GenRe benchmark [49] does not provide table data, the training data
for the table category are generated by randomly sampling 20 views from
ShapeNet meshes, following GenRe’s procedure. Within each example, we present
the real RGB-D scans and ground-truths from the input views.

Figure 8 shows results on two views of two different chairs and tables. For
chair scans, the left example shows the front view, and the completion results
only vary slightly across different runs. The right chair example shows a back
view, and the uncertainty allows more varied completion. Similarly, the left
table scan shows a large part of the table, so the completion varies less than
the right example, which only shows the top of the table.

##  5 Conclusion

We have introduced PVD, a unified framework for both shape generation and
shape completion. Our model, trained on a simple $\mathcal{L}_{2}$ loss, is
based on diffusion probabilistic models and learns to reverse a diffusion
process by progressively removing noise from noise-initialized samples. A
minor modification on the objective also results in a shape completion model
without the need for any architectural change. Experimentally, we show the
failure with straight-forward extension of diffusion models to either pure
voxel or point representations. With the point-voxel representation, our model
demonstrates superior generative power and impressive shape completion
quality. Unlike most baseline models which use deterministic encoder-decoder
structures, PVD can output multiple possible completion results given a
partial shape. Additionally, it can complete real 3D scans, thus offering
practical usage in various downstream applications.

#### Acknowledgements:

Yilun Du is funded by an NSF graduate fellowship. This work is in part
supported by ARMY MURI grant W911NF-15-1-0479, Samsung Global Research
Outreach (GRO) program, Amazon Research Award (ARA), IBM, Stanford HAI, and
Autodesk.

## References

  * [1] Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas.  Learning representations and generative models for 3D point clouds.  In ICML, 2018.
  * [2] Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston.  Generative and discriminative voxel modeling with convolutional neural networks.  arXiv:1608.04236, 2016.
  * [3] Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan.  Learning gradient fields for shape generation.  In ECCV, 2020.
  * [4] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al.  ShapeNet: An information-rich 3D model repository.  arXiv:1512.03012, 2015.
  * [5] Sungjoon Choi, Qian-Yi Zhou, Stephen Miller, and Vladlen Koltun.  A large dataset of object scans.  arXiv:1602.02481, 2016.
  * [6] Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese.  3D-R2N2: A unified approach for single and multi-view 3D object reconstruction.  In ECCV, 2016.
  * [7] Özgün Çiçek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger.  3D U-Net: Learning dense volumetric segmentation from sparse annotation.  In MICCAI, 2016.
  * [8] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio.  Density estimation using real NVP.  In ICLR, 2017.
  * [9] Yilun Du and Igor Mordatch.  Implicit generation and modeling with energy based models.  In NeurIPS, 2019.
  * [10] Haoqiang Fan, Hao Su, and Leonidas J. Guibas.  A point set generation network for 3D object reconstruction from a single image.  In CVPR, 2017.
  * [11] Matheus Gadelha, Rui Wang, and Subhransu Maji.  Multiresolution tree networks for 3D point cloud processing.  In ECCV, 2018.
  * [12] Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan Russell, and Mathieu Aubry.  AtlasNet: A Papier-Mâché approach to learning 3D surface generation.  In CVPR, 2018.
  * [13] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  In NeurIPS, 2020.
  * [14] Wenlong Huang, Brian Lai, Weijian Xu, and Zhuowen Tu.  3D volumetric modeling with introspective neural networks.  In AAAI, 2019.
  * [15] Aapo Hyvärinen.  Estimation of non-normalized statistical models by score matching.  JMLR, 6:695–709, 2005.
  * [16] Hyeongju Kim, Hyeonseung Lee, Woo Hyun Kang, Joun Yeop Lee, and Nam Soo Kim.  SoftFlow: Probabilistic framework for normalizing flow on manifolds.  In NeurIPS, 2020.
  * [17] Roman Klokov, Edmond Boyer, and Jakob Verbeek.  Discrete point flow networks for efficient point cloud generation.  In ECCV, 2020.
  * [18] Andrey Kurenkov, Jingwei Ji, Animesh Garg, Viraj Mehta, JunYoung Gwak, Christopher Choy, and Silvio Savarese.  DeformNet: Free-form deformation network for 3D shape reconstruction from a single image.  In WACV, 2018.
  * [19] Truc Le and Ye Duan.  PointGrid: A deep network for 3D shape understanding.  In CVPR, 2018.
  * [20] Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang.  A tutorial on energy-based learning.  Predicting Structured Data, 2006.
  * [21] Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov.  Point cloud GAN.  In ICLR Workshop, 2019.
  * [22] Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen.  PointCNN: Convolution on x-transformed points.  In NeurIPS, 2018.
  * [23] Minghua Liu, Lu Sheng, Sheng Yang, Jing Shao, and Shi-Min Hu.  Morphing and sampling network for dense point cloud completion.  In AAAI, 2020.
  * [24] Zhijian Liu, Haotian Tang, Yujun Lin, and Song Han.  Point-voxel cnn for efficient 3D deep learning.  In NeurIPS, 2019.
  * [25] Shitong Luo and Wei Hu.  Diffusion probabilistic models for 3D point cloud generation.  In CVPR, 2021.
  * [26] Daniel Maturana and Sebastian Scherer.  VoxNet: A 3D convolutional neural network for real-time object recognition.  In IROS, 2015.
  * [27] Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, and Hao Su.  PartNet: A large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding.  In CVPR, 2019.
  * [28] Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Ying Nian Wu.  Learning non-convergent non-persistent short-run MCMC toward energy-based model.  In NeurIPS, 2019.
  * [29] George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan.  Normalizing flows for probabilistic modeling and inference.  JMLR, 22(57):1–64, 2021.
  * [30] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas.  PointNet: Deep learning on point sets for 3D classification and segmentation.  In CVPR, 2017.
  * [31] Charles R Qi, Hao Su, Matthias Nießner Angela Dai Mengyuan Yan, and Leonidas J Guibas.  Volumetric and multi-view CNNs for object classification on 3D data supplementary material.  In CVPR, 2016.
  * [32] Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas.  PointNet++: Deep hierarchical feature learning on point sets in a metric space.  In NeurIPS, 2017.
  * [33] Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li.  PV-RCNN: Point-voxel feature set abstraction for 3D object detection.  In CVPR, 2020.
  * [34] Dong Wook Shu, Sung Woo Park, and Junseok Kwon.  3D point cloud generative adversarial network based on tree structured graph convolutions.  In ICCV, 2019.
  * [35] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In ICML, 2015.
  * [36] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  In ICLR, 2021.
  * [37] Yongbin Sun, Yue Wang, Ziwei Liu, Joshua E Siegel, and Sanjay E Sarma.  PointGrow: Autoregressively learned point cloud generation with self-attention.  In WACV, 2020.
  * [38] Kevin Swersky, Marc’Aurelio Ranzato, David Buchman, Benjamin M Marlin, and Nando de Freitas.  On autoencoders and score matching for energy based models.  In ICML, 2011.
  * [39] Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox.  Octree generating networks: Efficient convolutional architectures for high-resolution 3D outputs.  In ICCV, 2017.
  * [40] Diego Valsesia, Giulia Fracastoro, and Enrico Magli.  Learning localized generative models for 3D point clouds via graph convolution.  In ICLR, 2019.
  * [41] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon.  Dynamic graph CNN for learning on point clouds.  ACM TOG, 38(5):1–12, 2019.
  * [42] Zongji Wang and Feng Lu.  VoxSegNet: Volumetric cnns for semantic part segmentation of 3D shapes.  IEEE TVCG, 26(9):2919–2930, 2020.
  * [43] Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T Freeman, and Joshua B Tenenbaum.  Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling.  In NeurIPS, 2016.
  * [44] Rundi Wu, Xuelin Chen, Yixin Zhuang, and Baoquan Chen.  Multimodal shape completion via conditional generative adversarial networks.  In ECCV, 2020.
  * [45] Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Zhu Song-Chun, and Ying Nian Wu.  Learning descriptor networks for 3D shape synthesis and analysis.  In CVPR, 2018.
  * [46] Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan.  PointFlow: 3D point cloud generation with continuous normalizing flows.  In ICCV, 2019.
  * [47] Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian.  FoldingNet: Point cloud auto-encoder via deep grid deformation.  In CVPR, 2018.
  * [48] Maciej Zamorski, Maciej Zięba, Piotr Klukowski, Rafał Nowak, Karol Kurach, Wojciech Stokowiec, and Tomasz Trzciński.  Adversarial autoencoders for compact representations of 3D point clouds.  CVIU, 193:102921, 2020.
  * [49] Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Joshua B Tenenbaum, William T Freeman, and Jiajun Wu.  Learning to reconstruct shapes from unseen classes.  In NeurIPS, 2018.

Appendix: 3D Shape Generation and Completion through Point-Voxel Diffusion

##  Appendix A Additional Generation Metrics

We present additional generation metrics in Table 4, following PointFlow [46].
We report coverage (COV), which measures the fraction of point clouds in the
reference set that are matched to at least one point cloud in the generated
set. We further report minimum matching distance (MMD), which measures for
each point cloud in a reference set, the distance to its nearest neighbor in
the generated set. Note that these generation metrics can vary depending on
implementation and do not necessarily correlate to generation quality, as
discussed in [46].

Table 5 includes generation results on Airplane, Chair, Car compared with the
voxel-diffusion model, Vox-Diff, as described in the main paper, evaluated
using the 1-NN metric. By generating less noisy point clouds, PVD
significantly outperforms Vox-Diff.

Model |  | Airplane | Chair | Car |
---|---|---|---|---|---
MMD$\downarrow$ |  COV$\uparrow$ (%) |  MMD$\downarrow$ |  COV$\uparrow$ (%) |  MMD$\downarrow$ |  COV$\uparrow$ (%)
CD | EMD | CD | EMD | CD | EMD | CD | EMD | CD | EMD | CD | EMD
r-GAN [1] | 0.4471 | 2.309 | 30.12 | 14.32 | 5.151 | 8.312 | 24.27 | 15.13 | 1.446 | 2.133 | 19.03 | 6.539
l-GAN (CD) [1] | 0.3398 | 0.5832 | 38.52 | 21.23 | 2.589 | 2.007 | 41.99 | 29.31 | 1.532 | 1.226 | 38.92 | 23.58
l-GAN (EMD) [1] | 0.3967 | 0.4165 | 38.27 | 38.52 | 2.811 | 1.619 | 38.07 | 44.86 | 1.408 | 0.8987 | 37.78 | 45.17
PointFlow [46] | 0.2243 | 0.3901 | 47.90 | 46.41 | 2.409 | 1.595 | 42.90 | 50.00 | 0.9010 | 0.8071 | 46.88 | 50.00
SoftFlow [16] | 0.2309 | 0.3745 | 46.91 | 47.90 | 2.528 | 1.682 | 41.39 | 47.43 | 1.187 | 0.8594 | 42.90 | 44.60
DPF-Net [17] | 0.2642 | 0.4086 | 46.17 | 48.89 | 2.536 | 1.632 | 44.71 | 48.79 | 1.129 | 0.8529 | 45.74 | 49.43
Shape-GF [3] | 2.703 | 0.6592 | 40.74 | 40.49 | 2.889 | 1.702 | 46.67 | 48.03 | 9.232 | 0.7558 | 49.43 | 50.28
Vox-Diff | 1.322 | 0.5610 | 11.82 | 25.43 | 5.840 | 2.930 | 17.52 | 21.75 | 5.646 | 1.551 | 6.530 | 22.15
PVD (Ours) | 0.2243 | 0.3803 | 48.88 | 52.09 | 2.622 | 1.556 | 49.84 | 50.60 | 1.077 | 0.7938 | 41.19 | 50.56
Table 4: Additional generation metrics, following PointFlow [46]. | Airplane | Chair | Car
---|---|---|---
| CD | EMD | CD | EMD | CD | EMD
Vox-Diff | 99.75 | 98.13 | 97.12 | 96.74 | 99.56 | 96.83
PVD (ours) | 73.82 | 64.81 | 56.26 | 53.32 | 54.55 | 53.83
Table 5: Generation results on Airplane, Chair, Car compared with the voxel-
diffusion model, Vox-Diff, as described in the main paper, evaluated using the
1-NN metric. By generating less noisy point clouds, PVD significantly
outperforms Vox-Diff.

##  Appendix B Point Cloud Generation Visualization

We additionally visualize some generation results for Airplane, Car, and Chair
in terms of the generation process and the final generated shapes from all
angles in Figures 11, 12, 13. 14, 15, 16, 17, and 18.

##  Appendix C Derivation of the Variational Lower Bound

$\displaystyle{\rm E}_{q({\bf x}_{0})}[\log p_{\theta}({\bf x}_{0})]$
$\displaystyle={\rm E}_{q({\bf x}_{0})}\Big{[}\log\int p_{\theta}({\bf
x}_{0},...,{\bf x}_{T})\,d{\bf x}_{1:T}\Big{]}$ $\displaystyle\geq{\rm
E}_{q({\bf x}_{0})}\Big{[}\int q({\bf x}_{1},...,{\bf x}_{T}|{\bf x}_{0})$
$\displaystyle\log\frac{p_{\theta}({\bf x}_{0},...,{\bf x}_{T})}{q({\bf
x}_{1},...,{\bf x}_{T}|{\bf x}_{0})}\,d{\bf x}_{1:T}\Big{]}$
$\displaystyle={\rm E}_{q({\bf x}_{0:T})}\Big{[}\log\frac{p_{\theta}({\bf
x}_{0},...,{\bf x}_{T})}{q({\bf x}_{1},...,{\bf x}_{T}|{\bf x}_{0})}\Big{]},$

where the inequality is by Jensen’s inequality.

##  Appendix D Properties of the Diffusion Model

$\\{\beta_{0},...,\beta_{T}\\}$ is a sequence of increasing parameters;
$\alpha_{t}=1-\beta_{t}$ and $\tilde{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$.
Two following properties are crucial to deriving the final $\mathcal{L}_{2}$
loss.

Property 1. Tractable marginal of the forward process:

$\displaystyle q({\bf x}_{t}|{\bf x}_{0})$ $\displaystyle=\int q({\bf
x}_{1:t}|{\bf x}_{0})\,d{\bf x}_{1:{(t-1)}}$
$\displaystyle={\mathcal{N}}(\sqrt{\tilde{\alpha}_{t}}{\bf
x}_{0},(1-\tilde{\alpha}_{t}){\bf I}).$

This property is proved in the Appendix of [13] and provides convenient
closed-form evaluation of ${\bf x}_{t}$ knowing ${\bf x}_{0}$:

${\bf x}_{t}=\sqrt{\tilde{\alpha}_{t}}{\bf
x}_{0}+\sqrt{1-\tilde{\alpha}_{t}}\bm{\epsilon},$ (11)

where $\bm{\epsilon}\sim{\mathcal{N}}(0,{\bf I})$.

Property 2. Tractable posterior of the forward process. We first note the
Bayes’ rule that connects the posterior with the forward process,

$q({\bf x}_{t-1}|{\bf x}_{t},{\bf x}_{0})=\frac{q({\bf x}_{t}|{\bf
x}_{t-1},{\bf x}_{0})q({\bf x}_{t-1}|{\bf x}_{0})}{q({\bf x}_{t}|{\bf
x}_{0})}.$

Since the three probabilities on the right are Gaussian, the posterior is also
Gaussian, given by

$\displaystyle q({\bf x}_{t-1}|{\bf x}_{t},{\bf x}_{0})=$ (12)
$\displaystyle{\mathcal{N}}(\frac{\sqrt{\tilde{\alpha}_{t-1}}\beta_{t}}{1-\tilde{\alpha}_{t}}{\bf
x}_{0}+\frac{\sqrt{\alpha_{t}}(1-\tilde{\alpha}_{t-1})}{1-\tilde{\alpha}_{t}}{\bf
x}_{t},\frac{(1-\tilde{\alpha}_{t-1})}{1-\tilde{\alpha}_{t}}\beta_{t}{\bf
I}).$

##  Appendix E Derivation of $\mathcal{L}_{2}$ Loss

We need to match generative transition $p_{\theta}({\bf x}_{t-1}|{\bf x}_{t})$
with ground-truth posterior $q({\bf x}_{t-1}|{\bf x}_{t},{\bf x}_{0})$, both
of which are Gaussian with a pre-determined variance schedule
$\beta_{1},...,\beta_{T}$. Therefore, maximum likelihood learning is reduced
to simple $\mathcal{L}_{2}$ loss of the form with two cases:

$\displaystyle
L_{t}=\begin{cases}\left\lVert\frac{\sqrt{\tilde{\alpha}_{t-1}}\beta_{t}}{1-\tilde{\alpha}_{t}}{\bf
x}_{0}+\frac{\sqrt{\alpha_{t}}(1-\tilde{\alpha}_{t-1})}{1-\tilde{\alpha}_{t}}{\bf
x}_{t}-\mu_{\theta}({\bf x}_{t},t)\right\rVert^{2},&t>1\\\\[10.0pt]
\left\lVert{\bf x}_{0}-\mu_{\theta}({\bf
x}_{t},t)\right\rVert^{2},&t=1\end{cases}$

where $\alpha_{t}=1-\beta_{t}$ and
$\tilde{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$. The supervision target of case
$t>1$ comes from Eqn. 12. We can Further reduce the case when $t>1$ by
substituting ${\bf x}_{0}$ as an expression of ${\bf x}_{t}$ using Eqn. 11 and
arrive at

$\mathcal{L}_{t}=\begin{cases}\left\lVert\frac{1}{\sqrt{\alpha_{t}}}\left({\bf
x}_{t}-\frac{\beta_{t}}{\sqrt{1-\tilde{\alpha}_{t}}}\bm{\epsilon}\right)-\mu_{\theta}({\bf
x}_{t},t)\right\rVert^{2},&t>1\\\\[10.0pt] \left\lVert{\bf
x}_{0}-\mu_{\theta}({\bf x}_{t},t)\right\rVert^{2},&t=1\end{cases}$

where $\bm{\epsilon}\sim{\mathcal{N}}(0,{\bf I})$.

Note that when $t=1$, $\tilde{\alpha}_{1}=\alpha_{1}$ so that the supervision
target of the first case above evaluated at $t=1$ becomes:

$\displaystyle\frac{1}{\sqrt{\alpha_{1}}}\left({\bf
x}_{1}-\frac{\beta_{t}}{\sqrt{1-\tilde{\alpha}_{1}}}\bm{\epsilon}\right)$
$\displaystyle=\frac{1}{\sqrt{\tilde{\alpha}_{1}}}\left({\bf
x}_{1}-\sqrt{1-\tilde{\alpha}_{1}}\bm{\epsilon}\right)$ (13)
$\displaystyle={\bf x}_{0},$

where the last equality is by rewriting Eqn. 11. Therefore, in fact, the two
cases are equivalent.

The final $\mathcal{L}_{2}$ loss is

$\mathcal{L}_{t}=\left\lVert\frac{1}{\sqrt{\alpha_{t}}}\left({\bf
x}_{t}-\frac{\beta_{t}}{\sqrt{1-\tilde{\alpha}_{t}}}\bm{\epsilon}\right)-\mu_{\theta}({\bf
x}_{t},t)\right\rVert^{2}.$

Since ${\bf x}_{t}$ is known when ${\bf x}_{0}$ is known, we can redefine the
model output as $\bm{\epsilon}_{\theta}({\bf x}_{t},t)$. Instead of directly
predicting $\mu_{\theta}({\bf x}_{t},t)$, we instead predict a noise value
$\epsilon_{\theta}({\bf x}_{t},t)$, where

$\mu_{\theta}({\bf x}_{t},t)=\frac{1}{\sqrt{\alpha_{t}}}\left({\bf
x}_{t}-\frac{\beta_{t}}{\sqrt{1-\tilde{\alpha}_{t}}}\bm{\epsilon}_{\theta}({\bf
x}_{t},t)\right).$ (14)

Substituting in $\mu_{\theta}({\bf x}_{t},t)$ into the loss, we can arrive at
the final loss

$\left\lVert\bm{\epsilon}-\bm{\epsilon}_{\theta}({\bf
x}_{t},t)\right\rVert^{2},\;\;\bm{\epsilon}\sim{\mathcal{N}}(0,{\bf I}).$ (15)

##  Appendix F Point Cloud Generation Process

Since the transition mean $\mu_{\theta}({\bf x}_{t},t)$ of $p_{\theta}({\bf
x}_{t-1}|{\bf x}_{t})$ is calculated by Eqn. 14, the generative process is
performed by progressively sampling from $p_{\theta}({\bf x}_{t-1}|{\bf
x}_{t})$ as $t=T,...,1$:

${\bf x}_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}\left({\bf
x}_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\tilde{\alpha}_{t}}}\bm{\epsilon}_{\theta}({\bf
x}_{t},t)\right)+\sqrt{\beta_{t}}\mathbf{z},$ (16)

where $\mathbf{z}\sim{\mathcal{N}}(0,{\bf I})$. This approach is also similar
to Langevin dynamics [9, 28] used in energy-based models, since it similarly
adds scaled noise outputs from the model to current samples. Specifically,
Langevin dynamics is in the following form:

${\bf x}_{t+1}={\bf x}_{t}+s\frac{\partial}{\partial{\bf x}}\log
p_{\theta}({\bf x})+\sqrt{2s}\mathbf{z},$ (17)

where $s$ denotes step size, $p_{\theta}({\bf x})$ denotes the model
distribution, and $\mathbf{z}\sim{\mathcal{N}}(0,{\bf I})$. Both processes are
Markovian, shifting the previous output by a model-dependent term and a noise
term. The scaled model output of our model can also be seen as an
approximation of gradients of an energy function.

##  Appendix G Controlled Completion

We show that our model can control the shape completion process in Figure 9.
Given a pretrained completion model, our formulation also enables control over
completion results through latent interpolation. The figure below shows one
such case: we may have used the left depth map to obtain a completed shape.
But it comes with an unwanted feature (cavity at the chair’s back). We can
refine the result by feeding another depth map (shown on the right), with a
better view of the back. To retain features from both partial shapes, we can
interpolate by $(1-\lambda)\hat{{\bf x}}_{T}+\lambda\hat{{\bf y}}_{T}$ in the
latent space at time $T$, where the latent features are obtained by $\hat{{\bf
x}}_{T}=\sqrt{\tilde{\alpha}_{T}}\hat{{\bf
x}}_{0}+\sqrt{1-\tilde{\alpha}_{T}}\epsilon$ (see Eqn. 11). Interpolation
presents diverse choices and users can actively control how much features are
shared by varying $\lambda$.

Figure 9: Controlled completion process.

##  Appendix H Training Details

###  H.1 Model Architecture

Same as in [24], our point-voxel CNN architecture is modified from PointNet++,
where we replace the PointNet substructure with point-voxel convolution, as
shown in Figure 10. We specify our architecture in Table 6, Table 7, and and
Table 8. Table 6 shows details of a single set abstraction (SA) module. Table
7 shows details of a single feature propagation (FP) module. Table 8 shows how
these modules are combined together.

In particular, we concatenate the temporal embeddings with point features
before sending input into the Set Abstraction or the Feature Propagation
modules. To obtain temporal embeddings, we used a sinusoidal positional
embedding, commonly used in Transformers. Given a time $t$ and an embedding
dimension $d$, the time embedding consists of pairs of $\sin$ and $\cos$ with
varying frequencies,
$(\sin{(\omega_{1}t)},\cos{(\omega_{1}t)},...,\sin{(\omega_{d/2}t)},\cos{(\omega_{d/2}t)})$,
where $\omega_{k}$ is $1/\left(10000^{2k/d}\right)$.

We use the same architecture for both generation and completion tasks. For
shape completion specifically, the model takes as input a 200-point partial
shape and 1,848 points sampled from noise, totaling 2048 points. At each step,
the first 200 of the 2,048 points sampled by the model are replaced with the
input partial shape. The updated point set is then used as input in the next
time step.

Figure 10: Model architecture diagram.  Set Abstraction
---
Input Feature Size: $N_{\text{input}}\times C_{\text{input}}$
Input Time Embedding Size: $N_{\text{input}}\times E_{t}$
Output Feature Size: $N_{\text{output}}\times C_{\text{output}}$
Voxelization Resolution: $D$
Number of Point-Voxel Convolution (PVConv) blocks: $L$
Whether to use attention mechanism: use_attn
PVConv $\times L$
Layers | In-Out Size | Stride
Input: (X, t) | ($D\times D\times D\times C_{\text{input}}$, $D\times D\times D\times E_{t}$) |
Concat(X, t) | - | -
3x3x3 conv($C_{\text{output}}$), GroupNorm(8), Swish | $D\times D\times D\times C_{\text{output}}$ | 1
Dropout(0.1) |  |
3x3x3 conv($C_{\text{output}}$), GroupNorm(8) | $D\times D\times D\times C_{\text{output}}$ | 1
Attention(use_attn) | $D\times D\times D\times C_{\text{output}}$ |
MLP
1x1x1 conv($C_{\text{output}}$), GroupNorm(8), Swish | $D\times D\times D\times C_{\text{output}}$ | 1
Sampling & Grouping
Number of Centers: $N_{\text{center}}$
Grouping Radius: $r$
Number of Neighbors: $N_{\text{neighbor}}$
Table 6: Set Abstraction Layer. Input is first fed through $L$ PVConv modules,
then to an MLP module, and finally through the Sampling & Grouping module.
Feature Propagation
---
Input Feature Size: $N_{\text{input}}\times C_{\text{input}}$
Output Feature Size: $N_{\text{output}}\times C_{\text{output}}$
Voxelization Resolution: $D$
Number of Point-Voxel Convolution (PVConv) blocks: $L$
Whether to use attention mechanism: use_attn
Interpolation
PVConv $\times L$
Layers | In-Out Size | Stride
Input: X | $D\times D\times D\times C_{\text{input}}$ |
3x3x3 conv($C_{\text{output}}$), GroupNorm(8), Swish | $D\times D\times D\times C_{\text{output}}$ | 1
Dropout(0.1) |  |
3x3x3 conv($C_{\text{output}}$), GroupNorm(8) | $D\times D\times D\times C_{\text{output}}$ | 1
Attention(use_attn) | $D\times D\times D\times C_{\text{output}}$ |
MLP
3x3x3 conv($C_{\text{output}}$), GroupNorm(8), Swish | $D\times D\times D\times C_{\text{output}}$ | 1
Table 7: Feature Propagation Layer. Input is fed through Interpolation module,
$L$ PVConv modules, and an MLP module. Input Feature Size: $2048\times 3$
---
Input Time Embedding Size: 64
Output Feature Size: $2048\times 3$
Time Embedding
Sinusoidal Embedding dim = 64
MLP(64, 64)
LeakyRelU(0.1)
MLP(64, 64)
| SA 1 | SA 2 | SA 3 | SA 4
$L$ | 2 | 3 | 3 | 0
$C_{\text{input}}$ | 3 | 32 | 64 | -
$E_{t}$ | 64 | 64 | 64 | -
$C_{\text{output}}$ | 32 | 64 | 128 | -
$D$ | 32 | 16 | 8 | -
use_attn | False | True | False | -
$N_{\text{center}}$ | 1024 | 256 | 64 | 16
$r$ | 0.1 | 0.2 | 0.4 | 0.8
$N_{\text{neighbor}}$ | 32 | 32 | 32 | 32
| FP 1 | FP 2 | FP 3 | FP 4
$L$ | 3 | 3 | 2 | 2
$C_{\text{input}}$ | 128 | 256 | 256 | 128
$C_{\text{output}}$ | 256 | 256 | 128 | 64
$D$ | 8 | 8 | 16 | 32
use_attn | False | True | False | False
MLP(64,3)
Table 8: Entire point-voxel CNN architecture. Input point clouds and time
steps are sequentially passed through SA 1-4, FP 1-4, and an MLP to obtain
output of the same dimension. At the start of each SA and FP module, time
embedding and point features are first concatenated.

###  H.2 Choices of $\beta_{t}$ and $T$

For both hyper-parameters, we follow [13]. For Car and Chair, we set
$\beta_{0}=10^{-4},\beta_{T}=0.01$ and linearly interpolate other $\beta$’s.
For Airplane, we interpolate between $\beta_{0}=10^{-5}$ and $\beta_{T}=0.008$
for the first 90% steps and then fix $\beta_{T}=0.008$. We also set $T=1000$
for all experiments and we generally notice that lower timesteps (_e.g_., 100)
are not enough for the model to construct shapes.

###  H.3 Training Parameters

We use Adam optimizer with learning rate $2\times 10^{-4}$ for all
experiments.

Figure 11: Airplane generation process. Figure 12: Airplane results from all
angles. Figure 13: Car generation process. Figure 14: Car results from all
angles. Figure 15: Chair generation process. Figure 16: Chair results from all
angles. Figure 17: Chair generation process. Figure 18: Chair results from all
angles.


Also at ]Lab-STICC - UMR 6285 CNRS, Technopole Brest-Iroise CS83818, 29238
Brest Cedex 03 FranceAlso at ]Lab-STICC - UMR 6285 CNRS, Technopole Brest-
Iroise CS83818, 29238 Brest Cedex 03 France

Also at ]Lab-STICC - UMR 6285 CNRS, Technopole Brest-Iroise CS83818, 29238
Brest Cedex 03 France

# High wave vector non-reciprocal spin wave beams

L. Temdie  [  V. Castel  [  IMT- Atlantique, Dpt. MO, Technopole Brest-Iroise
CS83818, 29238 Brest Cedex 03 France  C. Dubs  INNOVENT e.V.
Technologieentwicklung, Pruessingstrasse 27B, 07745 Jena, Germany  G. Pradhan
J. Solano  H. Majjad  R. Bernard  Y. Henry  M. Bailleul  IPCMS - UMR 7504 CNRS
Institut de Physique et Chimie des Matériaux de Strasbourg France  V. Vlaminck
<EMAIL_ADDRESS>[  IMT- Atlantique, Dpt. MO, Technopole
Brest-Iroise CS83818, 29238 Brest Cedex 03 France

###### Abstract

We report unidirectional transmission of micron-wide spin waves beams in a 55
nm thin YIG. We downscaled a chiral coupling technique implementing Ni80Fe20
nanowires arrays with different widths and lattice spacing to study the non-
reciprocal transmission of exchange spin waves down to $\lambda\approx$ 80 nm.
A full spin wave spectroscopy analysis of these high wavevector coupled-modes
shows some difficulties to characterize their propagation properties, due to
both the non-monotonous field dependence of the coupling efficiency, and also
the inhomogeneous stray field from the nanowires.

††preprint: AIP/123-QED

##  I INTRODUCTION

Non-reciprocity is an essential properties of today’s information systems [2].
The ability to inhibit signal flow in one direction while allowing it in the
reverse direction is crucial to either protect devices from reflection,
isolate transmitters and receivers in radar architecture, or even shield
qubits from its environment in quantum computers. Most of the current non-
reciprocal functionalities rely on the gyrotropic nature of the magnetization
dynamics in field-based ferrimagnetic systems (namely Yttrium Iron Garnet -
YIG), which tend to be large, and costly to assemble [3]. Future progress in
communication systems, and most critically in quantum technologies, rely
heavily on the possibility to miniaturize and integrate these non-reciprocal
devices [4].
The field of magnonics, which implements magnetic excitation called spin waves
-or their quanta magnons-, is actively involved in the search of non-
reciprocal scalable solutions[5]. Extensive research in the last decade
revealed the possibility to engineer both amplitude and frequency non-
reciprocity of spin waves in many different ways [6]. Firstly, the well-known
Damon-Eshbach (DE) configuration, where the equilibrium magnetization of a
thin film lies in the plane of the film and perpendicular to the wavevector
[7], displays non-reciprocal dynamic amplitude across the thickness for
oppositely traveling waves, which couple chirally to an excitation antenna
[8]. Moreover, this non-reciprocity was recently proven to be strongly
enhanced in the presence of magnetic nanostructures, where unidirectional
transmission of spin waves was achieved using the chiral coupling between the
FMR of Co nanowires and exchange spin waves in a thin YIG film [9, 10].
Secondly, small frequency non-reciprocity (namely f(k) $\neq$ f(-k)) was
demonstrated more recently in various systems, either with asymmetrical
surface anisotropies between top and bottom surfaces [11], or with the coupled
dynamics of ferromagnetic multilayers [12], or also with the interfacial
Dzyaloshinskii-Moriya interaction (DMI) of a ferromagnetic layer coupled to a
high spin-orbit material [13, 14, 15]. Additionally, asymmetric spin-wave
dispersion was recently predicted in non-planar geometries due to a
topographically induced dynamic dipolar effect [16]. Nevertheless, all these
frequency non-reciprocity effects only becomes significant when the magnons
wavelength reaches sub micrometric sizes. Lastly, non-reciprocal
functionalities have also been predicted very recently using an innovative
inverse design approach [17, 18].
In this communication, we further miniaturized the method of Chen et al. [9]
and demonstrate the possibility of shaping non-reciprocal spin wave beams in a
continuous thin YIG film. This paper is organized as follows: in Sec. II, we
present the sample design and the experimental protocol used to measure
broadband multi-modes spin waves transmission. In Sec. III, we present the
non-reciprocal spin wave beams spectra, and address through a spin wave
spectroscopy study the peculiarities encountered in shaping narrow spin wave
beams via this method.

Figure 1:  (a) Sketch of the NWA-FMR mediated chiral coupling mechanism. (b)
SEM image of device I, $w$=300 nm width and $a$=500 nm lattice constant of
Ni80Fe20 NWA grown on top of 55 nm thickYIG film, 80 nm Au-antennas grown on
top of YIG|Ni80Fe20. Red and Yellow color represent respectively antenna
($k_{CPW}$) and Ni80Fe20 NWA($k_{NW}$) excite mode . Figure 2:  (f,H) mapping
of the spin wave spectra obtained for device I for (a) $\Delta L_{11}$, (b)
$\Delta L_{21}$, and (c) $\Delta L_{12}$. (d) Spectra obtained at
$\mu_{0}$Hext=+15 mT from (b) and (d). (e) Zoom of (d) on the $k_{CPW}$
region, and (f) on the $k_{NW}$ region. (g) Dispersion relation for field
ranging from 50 mT to 269 mT.

##  II Experimental Set-Up

###  II.1 Sample design and fabrication

The chiral excitation of propagating spin waves is achieved by coupling the
ferromagnetic resonnance (FMR) of magnetic nanowires array (NWA) to a low
damping continuous film such as YIG. As illustrated in Fig.1-(a), the phase
profile of a propagating spin wave excited by the dynamic dipolar field of
nanowires, all precessing in phase, only matches for a single propagation
direction. The degree of chirality depends strongly on the ellipticity of the
dynamic dipolar field of the NWA [19]. Typically, the elliptical polarization
of the Kittel mode in flat rectangular nanowires breaks the perfect chirality.
For this reason we made the nanowires 60 nm thick to come as close as possible
to an aspect ratio $t/w=1$ which produces circularly polarized dipolar field.
Moreover, the Kittel mode frequency of the nanowires can be tuned up with
decreasing their width $w$, therefore the NWA can excite propagating modes
($k_{NW}$) with much higher wavevector than the one directly coupled by the
microwave field of the antenna ($k_{CPW}$). In the case of an extended array
of NWA, these high vector ($k_{NW}$) correspond simply to integer values of
$\frac{2\pi}{a}$ accordingly with the periodic boundary conditions of the
phase precession [20, 21], minus some possible extinction related to the ratio
of $w/a$. In order to study the dependence of the NWA array density on the
excitation efficiency of the spin wave beam, we designed two devices with
different width $w$ and lattice constant $a$. Namely, for device I, $w$=300 nm
and $a$=500 nm; and for device II, $w$=200 nm and $a$=400 nm. The NWA length
of 10 $\mu$m was kept the same for both devices. Such a localized distribution
of excitation field produces a focused emission of spin waves in a similar
fashion to the spin wave beam excited from a constricted CPW [22, 23]. In the
present geometry, the distance of propagation is well within the near-field
region defined as the ratio $L^{2}/\lambda_{NW}$, with $L$ being the antenna
length and $\lambda_{NW}$ the magnon wavelength, so that the near-field
diffraction pattern from such a localized excitation follows very closely its
shape. For this reason, the emission of $k_{NW}$ magnons remains confined
within the length of the NW, thus forming a spin wave beam of width closely
equal to 10 µm as sketched in Fig. 1-(b).
Fig.1-(b) shows a SEM image of device I. The sample fabrication required two
steps of e-beam lithography followed by e-beam evaporation lift-off process on
a 55 nm thick liquid phase epitaxy YIG film [24]. To circumvent the insulating
nature of the substrate, we resorted to an extra layer of conductive resist AR
Electra 92 [25] on top of PMMA. In the first step, we structured the 60 nm
thick Ni80Fe20 (Permalloy) using a 3 nm thin Ti adhesion layer, and a 8 nm Au
capping layer via e-beam evaporation. In the second step, we aligned for both
devices the same 20 $\mu$m long coplanar wave guides (CPW) with the following
lateral dimensions: 2 $\mu$m wide central line and 1 $\mu$m wide ground lines
each spaced by 1 $\mu$m with the central line. These pairs of spin wave
antenna made of 4 nm Ti + 80 nm Au also via ebeam evaporation. The center-to-
center distance D between the two CPW are respectively 20 $\mu$m for device I,
and 15 $\mu$m for device II.

###  II.2 Broadband Spin Wave Spectroscopy

We incorporated an home-made confined electromagnet onto a PM8 probe station
to perform VNA spin wave spectroscopy using a Rohde & Schwarz ZNA43GHz Vector
Network Analyzer, calibrated via a SOLT procedure with 150 $\mu$ m pitch
picoprobes. The sample is carefully placed within the 11 mm gap of the
electromagnet, whose poles are 15 mm in diameter, and produce an homogeneous
in-plane field along the poles axis (namely less than 0.3% variation at most)
up to 500 mT at 3 A. Due to the small hysteresis of the electromagnet, we
always initialize a measurement with a high current of 5A in order to be
consistent with the calibration of our magnet.
We performed broadband frequency sweep in the [0.5;20] GHz range at constant
applied field acquiring 3201 points with a resolution bandwidth of 100 Hz in
order to resolve widely spread-out multi-modes transmission spectra. We
measure 2-ports S-parameters for several field values, which we translate into
the corresponding $Z_{ij}$ impedance matrices. Furthermore, due to the small
area of NWA (namely 10*5$\mu$m2), the typical range of $S_{ij}$ signal
amplitudes are of the order of 10-4 in linear scale (or -80 dB in log scale),
while the noise floor lays at 10-6. We retrieve a flat base line by
subtracting the measurement at given field with a reference measurement at
another field value $H_{ref}$ far enough that there are no dynamic feature in
the frequency span. Finally, as the coupling of the spin wave to an antenna is
of inductive nature, we chose to represent our relative measurements in units
of inductance defined as [26, 27, 22]:

$\Delta L_{ij}(f,H)=\frac{1}{i\,2\pi\,f}(Z_{ij}(f,H)-Z_{ij}(f,H_{ref}))$ (1)

where the subscripts $(i,j)$=1 or 2 denote either a transmission measurement
between two antennas ($i\neq j$), or a reflection measurement done on the same
port ($i=j$).

##  III Experimental Results and discussion

###  III.1 Perfect non-reciprocal transmission

We present in Fig.2 a mapping in the (f,H) plane of the spectra $\Delta
L_{11}$, $\Delta L_{21}$, and $\Delta L_{12}$ acquired with device I for field
gradually changing from +46 mT to -46 mT. We observe a myriad of modes that we
can separated into two main regions. The first region at lower frequencies
corresponds to all the $k_{CPW}$ modes coupled directly by the microwave field
of the antenna, and which range typically from k$\approx$0 (namely the section
of the CPW around the picroprobes) to k$\approx$15 rad.$\mu$m-1 (the 8th
satellites peak of the antenna). As shown in Fig.2-(e) with the spectra
measured at $\mu_{0}$Hext=+15 mT, some partial non-reciprocity occurs between
$\Delta L_{21}$ and $\Delta L_{12}$ due to the ellipticity of the microwave
field of the antenna as mentioned above.
The second region above 8 GHz corresponds to the higher $k_{NW}$ modes
mediated by the FMR of the permalloy NWA. In this frequency region, Fig.2-(b)
and 2-(c) show a perfect 100% non-reciprocal transmission of spin waves.
Namely, at positive field values, $\Delta L_{21}$ shows large oscillations,
while no transmission occurs from port 2 to port 1. Conversely at negative
fields, the non-reciprocity is reversed, and no transmission occurs from port
1 to port 2. We emphasize that the dynamic range employed for our measurements
(iBW=100Hz, P=-10dBm) gives a noise floor of about 5 fH, while typical
transmission amplitude are of the order of 100 fH. Besides, one notices
between 0 and -15 mT that the transmission for $\Delta L_{12}$ is somewhat
dispersed, indicating that the magnetization of the permalloy NWA do not
conserve an anti-parallel orientation with the YIG magnetization, and that a
gradual switching of the Py NWA is likely occurring. Furthermore, as shown
with the spectra at +15 mT of Fig.2-(f), a closer look in this region shows an
additional perfectly non-reciprocal mode around 8.4 GHz on top of the more
pronounced mode at 9.4 GHz. Nevertheless, the frequency spacing between these
two modes is much smaller than expected, as the wavevector difference between
two adjacent modes, $k_{n+1}-k_{n}=\dfrac{2\pi}{a}$ would result in a
frequency difference of about 3 GHz, as can be deduced from Fig.2-(g). We
investigate further the nature of these $k_{NW}$ multimodes in the next
section.

Figure 3:  Transmission spectra at 37 mT and 163 mT for (a) device I and (c)
device II, blue and red line represent respectively transmission spectra
$\Delta L_{21}$ and $\Delta L_{12}$. Field dependence of the frequency for the
several mode of (b) device I and (d) device II. (e) Field dependence of all
the mode amplitudes. (f) and (g) Comparison of the measured group velocity
with theoretical expression.

###  III.2 High-k spin wave beam characterization

We make use of several distinct peaks to characterize the propagation of high
k-vector magnons on both devices I and II. To begin with, we use the Kittel
mode to fit the lower branch of the reflection spectra in Fig.2-(a), which
corresponds to the quasi k=0 FMR resonance of the YIG happening in the larger
portion of the CPW close by the picoprobe and far from the NWA. From this
analysis, we extract a value of the gyromagnetic ratio of
$\gamma/2\pi$=27.7$\pm 0.2$ GHz.T-1 and an effective magnetization
$\mu_{0}$Meff=185$\pm 1$ mT, which is identical to the saturation
magnetization Ms [24], suggesting no in-plane anisotropy. Then, we track the
small reflection peak visible in between the two regions (e.g. the peak at 7.4
GHz in Fig.2-(d)), which corresponds to the first perpendicular standing spin
wave (PSSW) mode, and we fit the difference of the square of the frequencies
between the PSSW and the Kittel modes [28]:

$\begin{split}f_{PSSW}^{2}-f_{FMR}^{2}=(\frac{\gamma}{2\pi}\mu_{0})^{2}\,[2M_{s}\Lambda^{2}\,\frac{\pi^{2}}{t^{2}}\,H_{ext}\\\
+M_{s}^{2}\,\Lambda^{2}\,\frac{\pi^{2}}{t^{2}}(1+\Lambda^{2}\frac{\pi^{2}}{t^{2}})]\end{split}$
(2)

where $\Lambda$=$\sqrt{\frac{2A_{ex}}{\mu_{0}\,M_{s}^{2}}}$ is the exchange
length. In doing so, we obtain the following exchange constant Aexch=3.85$\pm
0.1$ pJ.m -1. At last, we study the field dependence of the several $k_{NW}$
modes features. For this purpose, we use a simple Gaussian function multiplied
by a cosine to fit the oscillations of each transmission peak (see colored fit
in Fig.3-(a) and 3-(c)), which gives us the peak position, its amplitude, and
the period of oscillation. This leaves us fitting the field dependence of the
frequency of each mode with only the wavevector as fitting parameter.
We show in the Fig.3 the results of this methodology. For device I with a
lattice constant a=500 nm, we followed three modes which correspond to
$k_{a}^{I}$=61.3 rad.$\mu$m-1, $k_{b}^{I}$=66.7 rad.$\mu$m-1, and
$k_{c}^{I}$=74 rad.$\mu$m-1; and for device II with a=400 nm, we tracked two
modes $k_{a}^{II}$=72 rad.$\mu$m-1, and $k_{b}^{II}$=77 rad.$\mu$m-1.
It may seem curious at first that none of these wavevectors corresponds to an
integer value of $\dfrac{2\pi}{a}$. However, the Fourier transform of the NWA
dynamic dipolar field distribution, which gives the spectral efficiency of the
excitation [26], is rather complex to depict as it consists of the modulation
of the NWA periodicity with the field distribution of the CPW. We therefore
perceive these $k_{NW}$ modes to be neighboring ripples of a convoluted
spectral distribution that depends on the lattice periodicity, the width of
the nanowire, and the antenna field distribution.
We also reported the field dependence of the mode amplitudes in Fig.3-(e),
which shows non-monotonous behavior. As one can see in the spectra of
Fig.3-(a) and Fig.3-(c), the efficiency of the coupling to a particular mode
will be all the stronger that its frequency matches with the one of the NWA
FMR. As the gyromagnetic ratio of YIG is smaller than the one of permalloy
($\gamma_{Py}/2\pi$=29.5 GHz.T-1), the NWA dispersion will gradually cross the
$k_{NW}$ multimodes dispersion across the field range. Unfortunately, this
non-monotonous field dependence of the excitation efficiency makes it arduous
to characterize the attenuation of these high k-vector with just one kind of
device. A series of devices with different distance D would be more suited for
determining the attenuation length of these high-k SWB.
Finally, from the period of oscillation fosc of the transmission spectra, we
estimate the group velocity according to vg=fosc*D [22]. We compare our
measurement of vg with the theoretical expression in Fig.3-(f) and Fig.3-(g).
Although the agreement at lower wavevector is fair, we observe some
significant discrepancies in the group velocity among the $k_{NW}$ modes. We
foresee that an additional phase delay must occur through the propagation
path. Namely, considering that the length of NWA is comparable to the
propagation distance D, one can expect the static stray field of the wires to
cause some inhomogeneities in the static field distribution between the two
antennas.

##  IV CONCLUSION

We carried out a spin wave spectroscopy study on two different 50$\mu$m2
Ni80Fe20 nanowire arrays resonantly coupled to a continuous 55 nm thin YIG
film. We demonstrated unidirectional transmission of 10 $\mu$m wide spin wave
beams up to 77 rad.$\mu$m-1 in the [8;20] GHz frequency range. An attempt to
characterize the propagation properties of these high k-vector spin wave beams
reveals several peculiarities regarding the modes selection, their coupling
efficiency, and possible additional phase lag due to inhomogeneous stray field
from the nanowires. These findings serve as a guideline for future
miniaturization of nonreciprocal magnonic devices.

###### Acknowledgements.

The authors acknowledge the financial support from the French National
research agency (ANR) under the project MagFunc, the Region Bretagne with the
CPER-Hypermag project, and the Département du Finistère through the project
SOSMAG. We also want to thanks Bernard Abiven for the implementation of the
electromagnet and Guillaume Bourcin for fruitful discussions.

## Data Availability Statement

The data that support the findings of this study are available from the
corresponding author upon reasonable request.

## References

  * [1] .
  * [2] W. Palmer, et al., IEEE Micr. Magazine 20, 36 (2019).
  * [3] V. G. Harris, Modern microwave ferrites, IEEE Trans. Magn. 48, 3 (2012).
  * [4] M. Devoret, et al., Superconducting circuits for quantum information, Science 339, 1169 (2013).
  * [5] A. V. Chumak et al., IEEE TRANSACTIONS ON MAGNETICS 58, 6 (2022).
  * [6] A. Barman et al, J. Phys.: Condens. Matter 33, 413001 (2021).
  * [7] R. W. Damon and J. R. Eshbach, J. Phys. Chem. Solids 19, 308 (1961).
  * [8] R. Camley, Surf. Sci. Rep. 7,103 (1987).
  * [9] J. Chen, T. Yu, C. Liu, T. Liu, M. Madami, K. Shen, J. Zhang, S. Tu, M. S. Alam, K. Xia, M. Wu, G. Gubbiotti, Y. M. Blanter, G. E. W. Bauer, and H. Yu, Phys. Rev. B 100, 104427 (2019).
  * [10] H. Wang, et al., Nano Res. 14, 2133–2138 (2021).
  * [11] O. Gladii, M. Haidar,Y. Henry, M. Kostylev, and M. Bailleul, Phys. Rev. B 93, 054430 (2016).
  * [12] M. Grassi, M. Geilen, D. Louis, M. Mohseni, T. Brächer, M. Hehn, D. Stoeffler, M. Bailleul, P. Pirro, and Y. Henry, Phys. Rev.Appl. 14, 024047 (2020).
  * [13] J.-H. Moon, S.-M. Seo, K.-J. Lee, K.-W. Kim, J. Ryu, H.-W. Lee, R. D. McMichael, and M. D. Stiles, Spin-wave propagation in the presence of interfacial Dzyaloshinskii-Moriya interaction, Phys. Rev. B 88, 184404 (2013).
  * [14] T. Brächer, O. Boulle, G. Gaudin, and P. Pirro, Creation of unidirectional spin-wave emitters by utilizing interfacial Dzyaloshinskii-Moriya interaction, Phys. Rev. B 95, 064429 (2017)
  * [15] K. Di, V. L. Zhang, H. S. Lim, S. C. Ng, M. H. Kuo, J. Yu, J. Yoon, X. Qiu, and H. Yang, Phys. Rev. Lett. 114, 047201 (2015).
  * [16] J. A. Otalora, M. Yan, H. Schultheiss, R. Hertel, A. Kakay, Curvature-induced asymmetric spin-wave dispersion, Phys. Rev. Lett. 117, 227203 (2016).
  * [17] Q. Wang , A. V. Chumak, and P. Pirro, Nature Commun. 12, 2636 (2021).
  * [18] A. Papp, W. Porod, and G. Csaba, Nature Commun. 12, 6422 (2021).
  * [19] T. Yu, and G. E. W. Bauer, "Chiral Coupling to Magnetodipolar Radiation",Topics in Applied Physics Springer book series, vol. 138, (2021).
  * [20] C. Liu et al, Nature Comm. 9, 738 (2018).
  * [21] J. Chen, C. Liu, T. Liu, Y. Xiao, K. Xia, G. E. W. Bauer, M. Wu, and H. Yu, Phys. Rev. Lett. 120, 217202 (2018).
  * [22] N. Loayza, M. B. Jungfleisch, A. Hoffmann, M. Bailleul, and V. Vlaminck, Phys. Rev. B 98, 144430 (2018).
  * [23] H. S. Körner, J. Stigloher, C. H. Back, Phys. Rev. B 96, 100401(R) (2017).
  * [24] C. Dubs, O. Surzhenko, R. Thomas, J. Osten, T. Schneider, K. Lenz, J. Grenzer, R. Hübner, and E. Wendler, Phys. Rev. Materials 4, 024416 (2020).
  * [25] Protective Coating AR-PC 5091.02 (Electra 92), https://www.allresist.com/portfolio-item/protective-coating-ar-pc-5091-02-electra-92/.
  * [26] V. Vlaminck and M. Bailleul, Phys. Rev. B 81, 14425 (2010).
  * [27] O. Gladii, M. Collet, K. Garcia-Hernandez, C. Cheng, S. Xavier, P. Bortolotti, V. Cros, Y. Henry, J.-V. Kim, A. Anane, and M. Bailleul, Appl. Phys. Lett. 108, 202407 (2016).
  * [28] B. A. Kalinikos, and A. N. Slavin, J. Phys. C: Solid State Phys. 19, 7013 (1986).