Safe and reliable autonomy solutions are a critical component of next-generation intelligent transportation systems. Autonomous vehicles in such systems must reason about complex and dynamic driving scenes in real time and anticipate the behavior of nearby drivers. Human driving behavior is highly nuanced and specific to individual traffic participants. For example, drivers might display cooperative or non-cooperative behaviors in the presence of merging vehicles. These behaviors must be estimated and incorporated in the planning process for safe and efficient driving. In this work, we present a framework for estimating the cooperation level of drivers on a freeway and plan merging maneuvers with the drivers' latent behaviors explicitly modeled. The latent parameter estimation problem is solved using a particle filter to approximate the probability distribution over the cooperation level. A partially observable Markov decision process (POMDP) that includes the latent state estimate is solved online to extract a policy for a merging vehicle. We evaluate our method in a high-fidelity automotive simulator against methods that are agnostic to latent states or rely on a priori assumptions about actor behavior.

§ INTRODUCTION

Ensuring safe interactions between automated and human traffic participants is a crucial step to making autonomous driving in urban environments a reality. Autonomous vehicles (AVs) must reason about the evolution of highly stochastic driving scenarios to provide protection for human passengers. Predicting the behavior of other traffic participants is an essential step to modeling scene evolution. However, forecasting human behaviors is a challenging task, as myriad psychological and physiological factors determine a traffic participant's driving tendencies <cit.>. Furthermore, urban driving scenarios induce complex multi-agent interactions as traffic participants seek to complete individual objectives. For example, vehicles attempting to merge on a freeway must identify a gap in the traffic, while vehicles on the freeway may exhibit cooperative or non-cooperative behavior depending on their own latent characteristics. Consequently, reasoning about the intentions and future actions of agents is important for multi-agent driving interactions such as merging <cit.>, intersection navigation <cit.>, and freeway driving <cit.>.

Human driving behavior can be modeled using data-driven or rule-based methods. Data-driven models are trained on expansive driving datasets and are highly parameterized to capture nuanced driver characteristics. For example, generative adversarial networks and imitation learning techniques have been used to construct expressive models of human driving behavior <cit.>. However, data-driven approaches are black-box in nature and difficult to interpret. Furthermore, data-driven models might induce dangerous behavior in driving scenes that do not appear in the training dataset <cit.>. An alternative to data-driven models is rule-based driver models such as the Intelligent Driver Model <cit.>. Rather than extracting driving characteristics from data, rule-based models specify a mathematical model using a set of rules derived from expert knowledge. Although such models rely on assumptions of human driving characteristics and can fail to generalize across a range of scenarios, they are highly interpretable and provide useful insights into microscopic human driving behavior. <cit.> present an extensive survey of human driver models.

Once the parameters of a driver model have been specified, the behavior of traffic participants can be forecasted and input to a decision-making system such as open-loop planing, or model predictive control <cit.>. Game-theoretic frameworks <cit.> and reinforcement learning <cit.> have also been used for automotive decision making, but these strategies are computationally expensive. Partially observable Markov decision processes (POMDPs) are well-suited for AV decision making because they account for uncertainty in the environment and latent behaviors of traffic participants, and have been used with great success in the freeway driving context <cit.>. In this work, we model the merge scenario shown in <ref> as a POMDP and solve the POMDP online for a driving policy. Online driving strategies choose the next action using real-time sensor information, while offline driving strategies plan all actions in advance.

Several papers have investigated interaction-aware planning for merge scenarios. <cit.> considered merging on freeways and T-junctions; they fit parameters to an extension of the IDM using a nonlinear least-squares procedure. <cit.> and <cit.> conducted reinforcement learning-based techniques to learn merging policies. These works represent offline strategies for parameter estimation; we instead pursue an online strategy to capture the idiosyncrasies of individual drivers in real-time. The strategy presented by <cit.> is most similar to our approach; however, they only consider lane change merging. We investigate freeway merging, which has been qualitatively described as one of the riskiest and complex driving maneuvers frequently encountered in day-to-day driving <cit.>.

In this paper, we perform online parameter estimation for a rule-based driver model—the Cooperative Intelligent Driver Model (C-IDM) <cit.>—that explicitly models the cooperation level of vehicles on a main freeway lane encountering a merging vehicle. The estimated cooperation level informs an online POMDP-based planner that executes a merge maneuver with awareness of latent driver behaviors. Our specific contributions are:

* We formulate the latent behavior prediction task as a recursive Bayesian estimation problem and estimate the cooperation level of vehicles using a particle filter.

* We use the estimates of latent driver behaviors to inform an online POMDP-based planner which controls the merging vehicle. We thus account for two levels of uncertainty, as we propagate uncertainty over latent behavior through the planning framework.

* We evaluate the safety and efficiency of our driving strategy in a high-fidelity automotive simulator and benchmark our planning strategy against methods that do not model latent behaviors or rely on a priori assumptions about agent behavior.

§ DRIVER MODELS

Rule-based driver models can provide reasonable approximations of microscopic driver behavior with a relatively small number of parameters, making them ideally suited to the online parameter estimation framework. In this section we discuss the ubiquitous IDM <cit.> and an IDM extension for merge scenarios, the Cooperative IDM <cit.>. We also present approaches for learning driver model parameters.

§.§ Intelligent Driver Model

The Intelligent Driver Model is an adaptive cruise control model that governs longitudinal vehicle motion. The IDM specifies a mathematical model that balances an agent's desire to travel at a particular speed with the need to maintain adequate separation distance between a leader vehicle.

Given a current speed $v(t)$, a distance headway $d(t)$, and a desired speed $v_{\mathrm{des}}$, the IDM computes a desired acceleration according to

\begin{equation}
{a_{{\mathrm{IDM}}}} = {a_{\max }}\left( {1 - {{\left( {\frac{{v(t)}}{{{v_{{\mathrm{des}}}}}}} \right)}^4} - {{\left( {\frac{{{d_{{\mathrm{des}}}}}}{{d(t)}}} \right)}^2}} \right).
\end{equation}

The desired separation distance $d_{\mathrm{des}}$ is defined as

\begin{equation}
{d_{{\mathrm{des}}}} = {d_{{\mathrm{min}}}} + \tau \cdot v(t) - \frac{{v(t) \cdot r(t)}}{{2\sqrt {{a_{{\mathrm{max}}}} \cdot {b_{{\mathrm{pref}}}}} }}.
\end{equation}

Model inputs include the minimum allowable separation distance $d_{\mathrm{min}}$; the minimum allowable separation time $\tau$; the acceleration limit $a_{\mathrm{max}}$; and the deceleration limit $b_{\mathrm{pref}}$.

§.§ Cooperative Intelligent Driver Model

The Cooperative Intelligent Driver Model is an extension of the IDM developed by <cit.> for merge scenarios. The model governs the longitudinal acceleration of a vehicle on the main lane while considering the time to reach the merge point ($TTM$) for a merging agent. The C-IDM is parameterized by the IDM parameters as well as the cooperation level $c \in [0,1]$ that determines the extent to which the driver cooperates with the merging vehicle. A cooperation level of $c = 1$ corresponds to a cooperative agent that yields to the merging vehicle, while a cooperation level of $c = 0$ corresponds to a non-cooperative agent that ignores the merging vehicle.

Consider the merge scenario shown in <ref> with an ego vehicle $V_{\mathrm{E}}$, a leading agent on the main lane $V_{\mathrm{L}}$, and a trailing agent on the main lane $V_{\mathrm{T}}$. Let $TTM(V_{i})$ represent the time required for the $i$th actor to reach the specified merge point. C-IDM outputs the longitudinal acceleration for the trailing agent according to the following rule:

* If $TTM(V_{\mathrm{E}}) < c \times TTM(V_{\mathrm{T}})$, then $V_{\mathrm{T}}$ follows IDM while considering the projection of the ego on the main lane as its leading vehicle.

* If $TTM(V_{\mathrm{E}}) \geq c \times TTM(V_{\mathrm{T}})$ or if no merging vehicle exists, then $V_{\mathrm{T}}$ follows IDM while considering the next actor on the main lane, $V_{\mathrm{L}}$, as its leading vehicle.

§.§ Rule-Based Driver Model Parameter Estimation

Rule-based driver models are governed by a relatively small number of parameters, which can be estimated in either an offline or an online fashion. Offline methods typically aggregate driving data in the training dataset and thus lose information on individual traffic participants. However, such methods can use arbitrarily large computation times and dataset sizes to maximize accuracy. Conversely, online methods must operate with constraints on computation time and data availability but capture the behavior of individual drivers. Offline strategies for rule-based driver model parameter estimation include constrained nonlinear optimization <cit.>, the Levenberg-Marquardt algorithm <cit.>, and genetic algorithms <cit.>. Online strategies include recursive Bayesian filters such as the Extended Kalman filter <cit.> and the particle filter <cit.>. We employ a particle filter to conduct latent parameter estimation due to its ability to represent complex belief states.

§ METHODOLOGY

We model the merge scenario as a partially observable Markov decision process, which can be defined by the tuple $(\mathcal{S},\mathcal{A},T,R,\mathcal{O},Z,\gamma)$, with $\mathcal{S}$ the state space, $\mathcal{A}$ the action space, $T$ the transition model, $R$ the reward function, $\mathcal{O}$ the observation space, $Z$ the observation likelihood function, and $\gamma$ the discount factor. An agent in state $s \in \mathcal{S}$ takes action $a \in \mathcal{A}$ and transitions to a successor state $s^{\prime}$ according to the transition model $T(s^{\prime},s,a) = \mathrm{Pr}(s^{\prime} \mid s, a)$. The agent receives a reward according to the reward function $R(s,a,s^{\prime})$. Rather than observing the true state, the agent receives a potentially imperfect observation $o \in \mathcal{O}$ at each time step. The likelihood of receiving observation $o$ is governed by $Z(o,s,a) = \mathrm{Pr}(o \mid s, a)$. Finally, the discount factor $\gamma$ can be adjusted to make the agent more or less myopic. We model the merge scenario with the following definitions:

§.§ The State Space

The state of a given driving scene,

\begin{equation*} s=(V_{E},\ V_{T},\ c_{T},\ V_{L})\in \mathcal{S}, \end{equation*}

consists of the observable states $V_{E}$, $V_{T}$, $V_{L}$ of the ego, trailing agent, and lead agent, respectively, and the latent cooperation level $c_{T}$ which governs the C-IDM for the trailing agent. The observable vehicle state

\begin{equation*} V_{i}=(x_{i},\ y_{i},\ v_{i},\ \dot{v}_{i},\ \theta_{i}) \end{equation*}

consists of the $i$th vehicle's position $(x_{i},\ y_{i})$ on the simulator map, longitudinal velocity $v_{i}$, longitudinal acceleration $\dot{v}_{i}$, and heading angle $\theta_{i}$.

§.§ The Action Space

At each time step, the ego takes an action $a \in \mathcal{A}$ which consists of a longitudinal jerk value. Planning occurs in jerk space to produce smooth velocity profiles. Jerk values are restricted to the range $\SI{-0.6}{\meter/\second^3} \leq a \leq \SI{0.6}{\meter/\second^3}$ to prevent unrealistic or unsafe motion.

§.§ The Transition Model

The transition model is implicitly defined by a generative model, $s^{\prime} \sim T(s,a)$, which generates successor states from the distribution  $\mathrm{Pr}(s^{\prime} \mid s, a)$. Vehicles on the main lane propagate according to one-dimensional point mass dynamics:

\begin{align*} & {v^{t + 1}} = {v^t} + {\dot{v}^t}\Delta t \\ & {x^{t + 1}} = {x^t} + {v^t}\Delta t + \frac{1}{2}{\dot{v}^t}\Delta {t^2} \end{align*}

Lateral speed is assumed to be zero. The trailing agent's acceleration is governed by the C-IDM, while the lead agent moves with constant velocity. The ego vehicle propagates according to a state transition function

\begin{align*}  & {\dot{v}^{t + 1}} = {\dot{v}^t} + \ddot{v}^t\Delta t \\ & {v^{t + 1}} = {v^t} + {\dot{v}^t}\Delta t
\end{align*}

The applied jerk $\ddot{v}^t$ is the action $a$ extracted at the current time step. We assume that the ego travels a distance ${v^t}\Delta t$ along the merge lane curve during each discrete time step.

§.§ The Reward Function

Autonomous vehicles must balance multiple and occasionally competing objectives such as safety, control effort, and rider comfort. We construct the following reward function to promote safe and efficient merge maneuvers:

\begin{equation} R(s)= -\lambda_1 \Vert v_{E} - v_{\text{ref}}\Vert - \lambda_2 \Vert \dot{v}_{E} \Vert  - \lambda_3 \mathbf{1}_{b_{\text{hard}}} \end{equation}

where $v_{\text{ref}}$ is the ego vehicle's desired reference velocity and $\mathbf{1}_{b_{\text{hard}}}$ is an indicator function of hard braking:

\begin{equation*}
\mathbf{1}_{b_{\text{hard}}} = \begin{cases}
{1}&{\text{if}\;\text{$\Vert(x_E,\ y_E) - (x_T,\ y_T) \Vert < d_{\mathrm{safety}} $}}\\
\end{cases}
\end{equation*}

where $d_{\mathrm{safety}}$ is a user-defined safety threshold. We define hard braking as an uncomfortably abrupt deceleration that can be applied to prevent a collision. The hard brake rate is thus a proxy metric for AV safety and rider comfort across simulations. The weights $\lambda_i$ can be adjusted to modify the ego's preferences for safety, efficiency, and control effort.

§.§ Online Parameter Estimation

Given a sequence of observations of the trailing agent's position, we wish to obtain a distribution over the latent cooperation level $c_T$. We assume that the cooperation level is time-invariant during the brief interaction between the trailing agent and the merging ego. Our proposed approach frames the task of estimating latent driver behavior as a state estimation problem and solves the inference problem using recursive Bayesian estimation. The general form of the recursive Bayesian update equation is

\begin{equation}
p(\theta_T \mid \mathbf{y}_{1:t}) = \frac{p(\mathbf{y}_t \mid \theta_T)p(\theta_T \mid \mathbf{y}_{1:t-1})}
{\int_{\theta_T} p(\mathbf{y}_t \mid \theta_T)p(\theta_T \mid \mathbf{y}_{1:t-1}) \mathrm{d}\theta_T} \text{.}
\label{eqn:recursive_state_estimation}
\end{equation}

Because <ref> cannot be solved analytically <cit.>, we use particle filtering to provide an approximate solution to the inference problem. <Ref> presents our strategy for estimating the cooperation level of the trailing agent. The particles are initially sampled from a uniform distribution and are then propagated according to the POMDP transition model. To prevent particle deprivation, we add noise $d_{c_T}$ sampled from a discrete uniform distribution $D = \mathcal{U}(\{-0.05,\ 0,\ 0.05 \})$.

Input: Initial particle set $\Theta$, ego $V_E$, trailing agent $V_T$, leading agent $V_L$, POMDP model $\mathcal{P}$

$c_{T}^{0} \leftarrow 0.5$ (initialize cooperation parameter)

$t \gets 0, 1, \ldots, N\ \mathrm{(time \ steps)}$
$x_T^{t+1} \leftarrow \mathrm{true \ position \ of \ trailing \ agent}$

$i \leftarrow 1, 2, \ldots, K\ \mathrm{(particles)}$
$TTM(V_{\mathrm{E}}) < c \times TTM(V_{\mathrm{T}})$
$\dot{v}_{T}^{t+1} \leftarrow$ IDM with ego projection as leader

$\dot{v}_{T}^{t+1} \leftarrow$ IDM with lead agent as leader
Propagate vehicles according to $\mathcal{P}.T(s^{\prime}, a, s)$

$w_{i} \gets Z\left(x_T^{t+1} \ | \ x_{T,i}^{t+1}\right)$

$d_{c_T} \sim D$ (dithering)

$c_{T}^{t+1} \leftarrow c_{T}^{t} + d_{c_{T}}$, clamped between $0$ and $1$

$\Theta \leftarrow \mathrm{Resample \ from \ } \Theta \propto \left[ w_{1}, \ldots, w_{K}\right]$
Online Parameter Estimation

§ EXPERIMENTS

In this work, we consider a freeway merge scenario as visualized in <ref>. An ego vehicle—representing either an autonomous agent or a human-driven vehicle with a guardian software system—seeks a gap on the main freeway lane. The vehicles are positioned such that the projection of the ego vehicle is in front of the trailing agent. Furthermore, we induce a conflict between the ego and the trailing agent by selecting initial velocities that place them on an intercept trajectory at the merge point. <Ref> shows key values for the POMDP reward functions. The POMDP was solved online using partially observable Monte Carlo planning with observation widening (POMCPOW) <cit.>.

POMDP Reward Function Parameters
Parameter     Value

$\lambda_1$ (speed deviation)     $1$

$\lambda_2$ (control effort)    $1$

$\lambda_3$ (collision)     $100$

$d_{\mathrm{safety}}$     $\SI{15.0}{\meter}$

§.§ Benchmark Methods

We tested our approach for learning the cooperation parameter against two other POMDP-based planning approaches that make a priori assumptions about the trailing agent's cooperation level: first, a cautious approach where the trailing agent is always assumed to be non-cooperative (e.g., the ego always assumes $c = 0.0$), and second, a risky approach where the trailing agent is always assumed to be cooperative (e.g., the ego always assumes $c = 1.0$). We also benchmarked against a scenario wherein the ego acceleration was governed by the stochastic IDM (SIDM)—an extension of the IDM that outputs noisy acceleration values <cit.>. We reproduced the merge scene shown in <ref> using the Applied Intuition simulation engine and ran 500 trials for each driving strategy; the trailing agent was assigned a ground truth cooperation level of $c = 0.0$ in 250 trials and a ground truth cooperation level of $c = 1.0$ in the other 250 trials.

§.§ Filtering

<Ref> shows two examples of cooperation level parameter estimation drawn from the batch of experiments. The red line indicates the true cooperation level of the trailing agent while the blue line represents the ego's estimate of the cooperation level. The light blue regions show the 90% empirical error bounds. The parameter estimate is initialized to $c=0.5$ with diffuse error bounds. The ego vehicle observes the trailing agent either slow down to facilitate a merge (<ref>) or maintain its current speed while ignoring the ego (<ref>) and updates its estimate of the cooperation level accordingly. <Ref> indicates that the ego is able to quickly identify whether a trailing agent is behaving in a generally cooperative or uncooperative manner. The fact that the parameter estimates do not converge exactly to the true value is not unexpected; since the C-IDM relates the time to merge of the ego and the trailing agent with an inequality, there are in general a range of cooperation levels that will yield the same behavior from the trailing agent. A visualization of parameter estimation during merge scenarios was created using Applied Intuition simulation tools and can be found on the lab video channel at <https://youtu.be/KnM3azGH_Sg>.

Sample cooperation level estimation when the trailing agent is fully cooperative, e.g., $c = 1.0$

Sample cooperation level estimation when the trailing agent is fully non-cooperative, e.g., $c = 0.0$

Examples of online latent parameter estimation for the trailing agent's cooperation level. The estimate of $c$ quickly approaches the true value, indicating that the ego is aware of the trailing agent's cooperation level as it nears the merge point.

§.§ Performance Results

We count the occurrences of hard braking and divide by the total number of trials to obtain the hard brake rate across all experimental configurations. We also measure time-to-collision (TTC), which is a safety metric, and time-to-merge, which indicates how efficient a merge maneuver is.
Higher TTC values are associated with safer driving scenes <cit.>, while excessively high time-to-merge values could be indicative of the freezing robot problem wherein the robot freezes in place to avoid a collision <cit.>.

<Ref> displays the rate of hard brake incidents across all driving methods. Hard brake rates remain predictably low across experiments with a cooperative trailing agent since the trailing agent avoids conflict with the merging ego by considering its projection on the main lane. When the trailing agent is non-cooperative, an upper bound on performance is given by the POMDP-based planner that assumes full knowledge of the true cooperation level of $c = 0.0$. Our proposed method for learning $c$ and the strategy of fixing $c = 0.0$ complete all trials without a hard brake incident. Planning with a fixed $c = 1.0$ expresses overconfidence in the desire of the vehicle on the main lane to cooperate and results in a hard brake rate of over $65\%$. The SIDM driving strategy, which does not consider the cooperation level of the vehicle on the main lane, must conduct hard braking on every trial to avoid a collision.

Hard Brake Rate
Driving Strategy     2cHard Brake Rate (%)

$c_T = 1.0$     $c_T = 0.0$

Learned $c$     $0.0$     $0.0$

Fixed $c = 0.0$     $0.0$     $0.0$

Fixed $c = 1.0$     $0.4$     $65.2$

SIDM     $4.0$     $100.0$

<Ref> presents the results for the trials with a cooperative trailing agent. The POMDP-based driving strategies achieve average TTC values greater than $10$ seconds. Many SIDM trials also achieve high TTC values; however, the SIDM trials that result in hard brake incidents are associated with small TTC values. Our strategy for learning $c$ achieves comparable results to the planner that assumes full knowledge of the trailing agent's true cooperation level of $c = 1.0$. Note that assuming non-cooperative behavior does not underperform in these trials since the scenario is designed to induce conflict. In fact, assuming a non-cooperative agent leads to smaller time-to-merge values as the ego accelerates to reach the gap ahead of the trailing agent, as shown in <ref>.

Distribution over minimum time-to-collision values achieved in trials with a cooperative agent.

Distribution over time required to merge in trials with a cooperative agent.

Experimental results given a cooperative agent on the main lane. Our strategy for learning $c$ achieves comparable results to the nominal planner that assumes full knowledge of the trailing agent's true cooperation level of $c = 1.0$.

<Ref> displays the results for the trials with a non-cooperative trailing agent. Both the SIDM and fixed $c = 1.0$ methods suffered from elevated hard brake rates and correspondingly low average TTC values. Our proposed method and the nominal benchmark obtained by fixing $c = 0.0$ achieved higher TTC values as shown in <ref>. An explanation for the difference in TTC values is clear from <ref>. Our proposed approach and the nominal baseline experience decreased time-to-merge values as they accelerate into the gap in front of the trailing agent. The POMDP-based driving strategy that assumes cooperative behavior takes more time to merge and thus encounters a conflict with the trailing agent at the merge point. The SIDM approach does not consider the behavior of the trailing agent at all and thus takes the longest time to merge on average.

Distribution over minimum time-to-collision values achieved in trials with a non-cooperative agent.

Distribution over time required to merge in trials with a non-cooperative agent.

Experimental results given a non-cooperative agent on the main lane. Our strategy for learning $c$ achieves comparable results to the nominal planner that assumes full knowledge of the trailing agent's true cooperation level of $c = 0.0$.

§ CONCLUSION AND FUTURE WORK

This paper presented a method for conducting online merge planning with awareness of the latent behaviors of vehicles on the main lane. We modeled the behavior of agents on a freeway using the Cooperative Intelligent Driver Model and demonstrated accurate estimation of the latent cooperation level using a particle filter. Our estimate of the cooperation level informed a behavior-aware POMDP model that was solved online for a merging policy. The performance of the proposed strategy closely matched the results achieved by baseline strategies that assumed full knowledge of other agents' latent characteristics. Furthermore, our driving strategy greatly outperformed a rule-based driving strategy that did not consider the cooperation level of other agents. Although the application described in this paper is specific to freeway merge scenarios, the parameter estimation and online planning framework is quite general and can easily be extended to other driving scenarios or robotic applications.

This work assumed that the latent behavior of vehicles was time-invariant; future work will relax this assumption to consider agents that change their behavior during a driving scene.
Furthermore, the safety and efficiency of a merging ego vehicle will be improved by quantifying the position and intention uncertainty of agents on a freeway. Uncertainty-aware models of freeway agents will inform the construction of probabilistically safe ego trajectory envelopes.

§ ACKNOWLEDGMENTS

Toyota Research Institute (TRI) provided funds to assist the authors with their research, but this article solely reflects the opinions and conclusions of its authors and not TRI or any other Toyota entity. This research was supported in part by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-1656518. We thank Applied Intuition for support with the simulation platform.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

booktitleAmerican Control Conference (ACC)
titleOnline parameter estimation for human driver behavior

booktitleIEEE International Conference on Intelligent
Transportation Systems (ITSC)
titleCooperation-aware reinforcement learning for merging in dense

booktitleIEEE Intelligent Vehicles Symposium (IV)
titleBelief state planning for autonomously navigating urban

booktitleAmerican Control Conference (ACC)
titleThe value of inferring the internal state of traffic
participants for autonomous freeway driving

titleModeling human driving behavior through generative
adversarial imitation learning
journaltitlearXiv preprint arXiv:2006.06412

titleCongested traffic states in empirical observations and
microscopic simulations
journaltitlePhysical Review E

titleA taxonomy and review of algorithms for modeling and
predicting human driver behavior
journaltitlearXiv preprint arXiv:2006.08832

titleSafe driving envelopes for path tracking in autonomous
journaltitleControl Engineering Practice

booktitleInternational Symposium on Experimental Robotics
titleOn infusing reachability-based safety assurance within
probabilistic planning frameworks for human-robot vehicle interactions

titleGame-theoretic planning for self-driving cars in multivehicle
competitive scenarios
journaltitleIEEE Transactions on Robotics

booktitleIEEE International Conference on Robotics and Automation
titleReinforcement learning for autonomous driving with latent
state inference and spatial-temporal relationships

titleImproving automated driving through planning with human
internal states
journaltitlearXiv preprint arXiv:2005.14549

titleProbabilistic model for interaction aware planning in merge
journaltitleIEEE Transactions on Intelligent Vehicles

booktitleAmerican Control Conference (ACC)
titleComparison of parametric and non-parametric approaches for
vehicle speed prediction

titleAnalysis of recurrent neural networks for probabilistic
modeling of driver behavior
journaltitleIEEE Transactions on Intelligent Transportation

SAGE Publications Sage CA: Los Angeles, CA

titleCalibrating car-following models by using trajectory data:
Methodological study
journaltitleTransportation Research Record

booktitleIEEE International Conference on Intelligent
Transportation Systems (ITSC)
titleReal-time estimation of drivers' behaviour

titleA Hybrid Rule-Based and Data-Driven Approach to Driver
Modeling Through Particle Filtering
journaltitleIEEE Transactions on Intelligent Transportation

booktitleIEEE International Conference on Intelligent
Transportation Systems (ITSC)
titleInteraction-aware approach for online parameter estimation of
a multi-lane intelligent driver model

booktitleIEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS)
titleInteraction-aware probabilistic behavior prediction in urban

booktitleInternational Conference on Automated Planning and
Scheduling (ICAPS)
titleOnline algorithms for POMDPs with continuous state, action,
and observation spaces

titleThe intelligent driver model with stochasticity-new insights
into traffic flow oscillations
journaltitleTransportation Research Procedia

titleExtended time-to-collision measures for road traffic safety
journaltitleAccident Analysis & Prevention

booktitleIEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS)
titleUnfreezing the robot: Navigation in dense, interacting


# Hypernetworks build Implicit Neural Representations of Sounds

Filip Szatkowski  Karol J. Piczak  Przemysław Spurek  Jacek Tabor  Tomasz
Trzciński

###### Abstract

Implicit Neural Representations (INRs) are nowadays used to represent
multimedia signals across various real-life applications, including image
super-resolution, image compression, or 3D rendering. Existing methods that
leverage INRs are predominantly focused on visual data, as their application
to other modalities, such as audio, is nontrivial due to the inductive biases
present in architectural attributes of image-based INR models. To address this
limitation, we introduce HyperSound, the first meta-learning approach to
produce INRs for audio samples that leverages hypernetworks to generalize
beyond samples observed in training. Our approach reconstructs audio samples
with quality comparable to other state-of-the-art models and provides a viable
alternative to contemporary sound representations used in deep neural networks
for audio processing, such as spectrograms.

Machine Learning, Hypernetworks, Audio Processing, Implicit Neural
Representations

ack

##  1 Introduction

Figure 1: We propose HyperSound, the first framework which produces Implicit
Neural Representations for unseen audio signals, eliminating the need for
retraining a model for each new example.

The field of Implicit Neural Representations (INRs) is a rapidly growing area
of research, which aims to obtain functional, coordinate-based representations
of multimedia signals through the use of neural networks. These
representations are decoupled from the spatial resolution, so the signal can
be resampled at any arbitrary frequency without incurring additional memory
storage requirements. Applications of INRs include super-resolution (Sitzmann
et al., 2020b; Mehta et al., 2021), compression (Sitzmann et al., 2020b; Mehta
et al., 2021), 3D rendering (Mildenhall et al., 2021), and missing data
imputation (Fons et al., 2022). Furthermore, INRs can be used to synthesise
data (Zuiderveld et al., 2021), and they show promise as an alternative to the
traditional feature extraction techniques used in neural networks (Dupont et
al., 2022).

INRs are a promising research direction in audio processing, yet so far their
applications in this domain remain limited. As the size of INR networks can be
significantly smaller than the size of the represented signal, they can be
used as a compression method (Strümpler et al., 2022). They also provide an
alternative to super-resolution algorithms such as ProGAN (Karras et al.,
2018), which often require hand-crafting complex architectures for selected
resolution. In contrast, in INRs the signal resolution is given by the
granularity of input coordinates, so a single network represents the signal
for any resolution we might require while remaining constant in size.
Moreover, as (Müller et al., 2022) show that INRs can be used as features in
other algorithms, this representation can be an alternative for current deep
audio processing models that primarily operate on features obtained from mel
spectrograms or raw audio waves (Purwins et al., 2019).

In practice, architectures typically used for INRs struggle with modeling high
frequencies inherent in audio signals (Rahaman et al., 2019) and require
training a new model for every incoming data point, which is wasteful and
greatly reduces their applicability. This issue is usually addressed by
obtaining generalizable INRs through meta-learning methods, such as
hypernetworks (Ha et al., 2016) – models that produce the weights for other
models. The hypernetwork framework can be leveraged to obtain generalizable
INRs for a variety of data types, including images (Sitzmann et al., 2020b;
Mehta et al., 2021; Klocek et al., 2019), point clouds (Mehta et al., 2021;
Spurek et al., 2020), videos (Mehta et al., 2021) and time series (Fons et
al., 2022). However, obtaining generalizable INRs for audio signals remains an
unsolved task due to the high dimensionality and variance of this type of
data, which makes the standard methods for this task insufficient.

In our work, we propose a method that learns a general recipe for creating
INRs for arbitrary audio samples not present in the training dataset. We adapt
the hypernetwork framework for sound processing by introducing a domain-
specific loss function and hypernetwork weight initialization. Additionally,
we investigate various variants of the hypernetwork framework in order to
achieve improved compression while maintaining audio quality. To our
knowledge, our method, named HyperSound, is the first application of
hypernetwork-based INR generation to the audio domain.

The main contributions of our work are:

  * •

We introduce HyperSound, to our knowledge the first INR method for audio
processing that builds functional representations of unseen audio signals,
which eliminates the need of retraining the INR for each new data point.

  * •

We show a successful adaptation of the hypernetwork paradigm to the audio
processing through domain-specific modifications such as the spectral loss
function.

  * •

We empirically evaluate various approaches of using hypernetworks to build
INRs for audio generation, e.g. by leveraging SIREN architecture or Fourier
mapping.

##  2 Related Works

###  2.1 Implicit Neural Representations (INRs)

Implicit Neural Representations (INRs) are continuous, coordinate-based
approximations of data obtained through neural networks. The decoupling of
representation from spatial resolution makes INRs attractive for a wide range
of applications in signal processing. However, functions represented by
standard MLP networks with ReLU activation function show a strong bias towards
low frequencies (Rahaman et al., 2019), which in practice make them infeasible
as representations of multimedia signals. To mitigate this bias, most INRs use
either Fourier features (Tancik et al., 2020) or SIREN networks (Sitzmann et
al., 2020b) with periodic activation functions. Later works improve SIREN
networks, introducing various forms of modulation to sinusoidal layers (Mehta
et al., 2021; Chan et al., 2021). INRs are currently used in many
applications, such as super-resolution (Sitzmann et al., 2020b; Mehta et al.,
2021), compression (Strümpler et al., 2022), 3D rendering (Mildenhall et al.,
2021), missing data imputation (Fons et al., 2022) or data generation
(Zuiderveld et al., 2021). However, typical INRs are trained for each data
point, and obtaining a representation for new signals requires retraining the
network from scratch.

###  2.2 Generalizable INRs

To obtain a way of generating INRs for new data without the need for model
retraining, various meta-learning approaches were proposed. (Sitzmann et al.,
2020a) leverage gradient-based meta-learning algorithms to solve the task of
learning an INR space for shapes. (Chen & Wang, 2022) employ a transformer
meta-learner to obtain INRs for unseen examples. Many methods of obtaining
generalizable INRs also involve the use of hypernetworks (Ha et al., 2016),
where one network (hypernetwork) generates the weights for another network
(target network). More broadly, hypernetworks are employed in many
applications, including model compression (Zhao et al., 2020), continual
learning (Von Oswald et al., 2019) or generating INRs. In particular,
hypernetworks were successfully used to generate INRs for images (Klocek et
al., 2019; Sitzmann et al., 2020b; Mehta et al., 2021), shapes (Spurek et al.,
2020; Sitzmann et al., 2020b), videos (Mehta et al., 2021) and time series
(Fons et al., 2022). To our knowledge, our work is the first application of
hypernetworks for generating audio INRs.

###  2.3 Deep neural networks for audio processing

First successful attempts at raw waveform processing with deep neural networks
were models such as WaveNet (Oord et al., 2016) and SampleRNN (Mehri et al.,
2016), but their autoregressive nature makes them slow and prone to
accumulation of errors. Later architectures such as ParallelWaveNet (Oord et
al., 2018), NSynth (Engel et al., 2017), MelGAN (Kumar et al., 2019) or SING
(Défossez et al., 2018) proposed non-autoregressive architectures for audio
generation. Recent autoencoder-based models such as RAVE (Caillon & Esling,
2021) or SoundStream (Zeghidour et al., 2021) are able to process high-
resolution signals in an end-to-end fashion, producing audio of very good
perceptual quality.

##  3 Method

Figure 2: Overview of the HyperSound framework. We use a single hypernetwork
model to produce distinct INRs based on arbitrary audio signals provided as
input.

Traditional digital representation of sound waves is a series of amplitude
values sampled at regular time intervals, which approximates a continuous real
function $x(t)$. Our goal is to obtain a meta-recipe for generating audio INRs
approximating such functions. While the creation of INRs for particular audio
samples can be successfully achieved with gradient descent, finding a general
solution is much harder due to the inherent complexity of audio time series.
Therefore, similar to (Klocek et al., 2019), we model such functions with
neural networks $T$ (target networks), parameterized by weights generated by
another neural network $H$ (hypernetwork). Our method, named HyperSound, can
be described as

$\begin{gathered}\bm{\theta}_{\bm{x}}=H(\bm{x}),\\\
\hat{x}(t)=T(t,{\bm{\theta}}_{\bm{x}}).\end{gathered}$ (1)

We show the overview of our method in Fig. 2. In the following sections, we
describe the architecture of the models used as Implicit Neural
Representations and of the hypernetwork. Then, we present the details required
for the successful application of the hypernetwork framework to the audio
domain.

###  3.1 Implicit Neural Representations

We consider two variants of INR networks, inspired by NeRF (Mildenhall et al.,
2021) and SIREN (Sitzmann et al., 2020b) papers. Both models have a single
input and a single output.

We refer to a NeRF-inspired network as a Fourier-mapped multilayer perceptron
(FMLP). The network consists of a positional encoding layer followed by dense
layers with biases and ReLU activation functions. Positional embedding vectors
$\bm{\gamma}$ are obtained as

$\displaystyle\bm{\gamma}(t)=\,\Bigl{[}$ $\displaystyle\sin(2^{0}\pi t),\,$
$\displaystyle\cos(2^{0}\pi t),$ (2) $\displaystyle\sin(2^{1}\pi t),\,$
$\displaystyle\cos(2^{1}\pi t),$ $\displaystyle\ldots,$
$\displaystyle\sin({2}^{L-1}\pi t),\,$ $\displaystyle\cos({2}^{L-1}\pi
t)\Bigr{]}$

where $t$ is the time coordinate and $L$ denotes the embedding size. We
normalize input coordinates to a range of $[-1,1]$ and use $L=10$.

SIREN network consists of dense layers with a sinusoidal activation function.
The output $\bm{y_{i}}$ of $i$-th SIREN layer is defined as:

$\bm{y_{i}}=\sin{(\omega_{i}\bm{x_{i}}\bm{W_{i}}^{T}+\bm{b_{i}})},$ (3)

where $x_{i}$ is the input of the layer, $\bm{W_{i}}$ and $\bm{b_{i}}$ are
layer weights and biases and $\omega_{i}$ is a scalar factor introduced in
(Sitzmann et al., 2020b) to enable learning of higher signal frequencies. In
practice, following the original paper, we set a distinct $\omega_{0}\neq 1$
for the first layer and keep the same shared value of $\omega_{i}$ for all
remaining layers.

(Benbarka et al., 2022) prove that FMLP networks are equal to single-layer
SIREN networks. However, in our experiments, we see significant differences in
the learning dynamics and representational capacity of both networks and thus
explore both architectures in our experiments.

###  3.2 Hypernetwork

Typical audio recordings contain several thousands of samples, while
hypernetworks generally use fully connected layers to produce target weights.
Since we want to use raw audio recordings as input, using dense layers to
process the signal directly leads to the explosion of the number of weights in
the hypernetwork. To avoid this problem, we use a convolutional encoder to
produce a latent, lower dimensional representation of the signal and then
process this representation with fully connected layers. The hypernetwork
outputs weights $\bm{\theta}$ that parameterise the target network.

We use an encoder based on SoundStream (Zeghidour et al., 2021). The fully
connected part of the hypernetwork is composed of six layers with biases and
ELU (Clevert et al., 2015) activation, where the last layer produces the
flattened weights of the target network. Input of the hypernetwork is
$32\,768$ samples of audio at a sampling rate of $22\,050$ Hz, the size of a
latent representation is $103\times 32$ and the layer sizes of the dense part
are $[400,768,768,768,768,400]$.

Table 1: Comparison of different variants of computing target network pre-activation layer output. Network type |  Instance specific weights | Shared weights | Pre-activation layer output
---|---|---|---
standard |  all layers | none | $\bm{y_{i}}=\bm{x}(\bm{W_{i}^{h}})^{T}+\bm{b_{i}^{h}}$
additive |  all layers | all layers | $\bm{y_{i}}=\bm{x}(\bm{W_{i}^{s}}+\bm{W_{i}^{h}})^{T}+\bm{b_{i}^{s}}+\bm{b_{i}^{h}}$
multiplicative |  all layers | all layers | $\bm{y_{i}}=\bm{x}(\bm{W_{i}^{s}}\cdot\bm{W_{i}^{h}})^{T}+\bm{b_{i}^{s}}\cdot\bm{b_{i}^{h}}$
shared |  some layers | some layers | $\bm{y_{i}}=\begin{cases}\bm{x}(\bm{W_{i}^{s}})^{T}+\bm{b_{i}^{s}},&\text{if layer shared}\\\ \bm{x}(\bm{W_{i}^{h}})^{T}+\bm{b_{i}^{h}},&\text{otherwise}\end{cases}$

###  3.3 Hypernetwork adaptation for audio data

Audio signals contain inherent high-frequency content, and neural networks in
general tend to focus on low-frequency features (Rahaman et al., 2019). Using
a hypernetwork to produce INR weights introduces additional complexity to the
optimization process. During training, small updates to the hypernetwork
weights might disproportionally affect the function represented by the output
weights, which makes the training process unstable. Moreover, the outputs of
the neural network will always contain some inherent noise, which affects the
high-frequency content more. Finally, in the case of training INRs,
hypernetworks tend to overfit the training data, as they usually require over-
parametrization to be able to solve a difficult problem of function
generation. Combined, those issues make the straightforward application of the
hypernetwork framework to our problem impossible.

To enable hypernetworks to model audio data, we introduce STFT loss function,
which counters the hypernetworks tendency to ignore high-frequencies. The
optimization process is described in detail in Section 3.4. The spectral loss
component is crucial to obtain the INRs of sufficient quality, as using only
standard reconstruction loss in the time domain leads to the network
collapsing to trivial solutions such as uniform noise or silence.
Additionally, we notice that our method tends to overfit the training dataset
and find that in order to make hypernetwork generalize to the unseen data we
have to introduce random data augmentations to the training process.

###  3.4 Optimization for audio reconstruction

We train the hypernetwork in a supervised fashion using backpropagation. To
obtain audio results that are more perceptually pleasant, we use a loss
function that penalizes the reconstruction error both in time and frequency
domains. Given an original recording $\bm{x}$ and its reconstruction
$\hat{\bm{x}}$ generated with a target network, we compute the loss function
as:

$L(\bm{x},\hat{\bm{x}})={\lambda}_{t}L_{t}(\bm{x},\hat{\bm{x}})+{\lambda}_{f}L_{f}(\bm{x},\hat{\bm{x}}),$
(4)

where $L_{t}$ is the L1 loss in time domain and $L_{f}$ is a multi-resolution
mel-scale STFT loss in frequency domain as introduced in ParallelWaveGAN
(Yamamoto et al., 2020). We use coefficients ${\lambda}_{t}$ and
${\lambda}_{f}$ to control the tradeoff between both losses.

For the STFT loss, we use $128$ mel bins and FFT sizes of
$[2048,1024,512,256,128]$ with matching window sizes and an overlap of $75$%.
We set ${\lambda}_{t}=1$ and ${\lambda}_{f}=1$. Using both loss functions
together allows us to optimize both quantitative metrics through L1 part of
the loss and the perceptual quality of generated audio through the STFT part.
We base this intuition on results from (Fons et al., 2022), which show that
the FFT loss is crucial for learning INRs for time series.

Ground truthFMLPSIREN Figure 3: Example of VCTK validation sample
reconstructed with HyperSound.

Neural networks are naturally biased towards low-frequency features, and we
notice that the target networks obtained with HyperSound suffer from the same
problem. Therefore, in some experiments we modify the STFT loss by frequency-
weighting the Fourier transforms of $\bm{x}$ and $\bm{\hat{x}}$. We compute
the weightings $w_{f_{i}}$ for frequency bins $f_{1},f_{2},...,f_{N}$ as:

$w_{f_{i}}=\frac{N\cdot i^{p}}{\sum_{j=0}^{N}j^{p}},$ (5)

where $N$ is the number of frequency bins and $p\geq 0$ is a parameter that
allows us to control the uniformity of this weighting. The weights are
normalized so that they always sum to $N$, and the minimum value of $p$ yields
a uniform weighting (no bias towards high frequencies). During training, we
gradually move from uniform weighting to target weighting in 500 epochs.

###  3.5 Alternative ways to leverage hypernetworks for INR generation

In the standard variant of our framework, the hypernetwork generates weights
and biases for target network layers and other parameters, such as frequencies
of positional encoding in FMLP or $\bm{\omega_{i}}$ in SIREN, remain fixed.
However, we also consider learning a shared, generic set of weights and biases
for target network layers and combining them with instance-specific weights
and biases generated by the hypernetwork in other ways. We refer to the three
variants of target networks explored in our work as shared, additive and
multiplicative. We show a comparison of all the approaches in Table 1.

With a shared variant of the target network, we explicitly learn a shared set
of weights $\bm{W_{i}^{s}}$ and biases $\bm{b_{i}^{s}}$ for some layers, and
use the hypernetwork only for generating instance-specific weights
$\bm{W_{i}^{h}}$ and biases $\bm{b_{i}^{h}}$ for the remaining layers. Target
networks’ outputs are then computed using either shared or instance-specific
set of weights, depending on the layer.

In the additive and multiplicative variants of the target network, for each
layer $i$ we explicitly learn shared weights $\bm{W_{i}^{s}}$ and biases
$\bm{b_{i}^{s}}$. The hypernetwork learns to generate separate, instance-
specific, weights $\bm{W_{i}^{h}}$ and biases $\bm{b_{i}^{h}}$. Both sets of
weights and biases are then either added or multiplied and the results are
used to compute layer output.

###  3.6 Hypernetwork weight initialization

Hypernetworks are very sensitive to the learning rate and require careful
weight initialization to ensure stable learning. Xavier (Glorot & Bengio,
2010) or Kaiming (He et al., 2015) weight initializations typically used for
deep neural networks are insufficient for hypernetworks, as we want to achieve
the corresponding initial distribution of generated weights, but we can affect
it only through the weights of the hypernetwork. To solve this issue, (Chang
et al., 2019) propose hypernetwork weight initialization scheme which leads to
the initial target network weights following the distribution given by either
Xavier or Kaiming initialization. We adopt this process for our FMLP network,
but find it insufficient for SIREN networks, as they require specific
initialization described by (Sitzmann et al., 2020b). Therefore, to simulate
SIREN initialization in the initial stage of training, we set the starting
weights for the last layer of the hypernetwork according to uniform
distribution with boundaries close to zero. and set the distribution of biases
to the one proposed in the SIREN paper. This makes the frequencies in the
subsequent target network layers grow slowly in the initial stages of network
training, which is crucial to stable learning with periodic activation
functions.

##  4 Experiments

Table 2: HyperSound reconstruction error on the VCTK validation set compared with other methods. a) We compare our method with the state-of-the-art reconstruction model RAVE. b) We test different variants of the FMLP target network with different sizes as measured by the compression ratio (CR). c) We compare the reconstruction error obtained with SIREN target networks of size equal to FMLP with CR 1 and various values of $\omega_{i}$ and $\omega_{0}$. d) We also show the upper bound of INR reconstruction obtained by training a single model for each data point, using both FMLP and SIREN networks. Model |  |  |  | MSE | LSD | SI-SNR | PESQ | STOI | CDPAM
---|---|---|---|---|---|---|---|---|---
Ideal metric behavior: |  |  |  | $\to 0$ | $\to 0$ | $\to 100$ | $\to 4.5$ | $\to 1$ | $\to 0$
a) RAVE (Caillon & Esling, 2021) |  |  |  | 0.049 | 0.99 | -23.50 | 1.75 | 0.87 | 0.35
b) HyperSound – FMLP |  | CR |  |  |  |  |  |  |
|  | 10$\times$ |  | 0.013 | 1.80 | -0.25 | 1.14 | 0.71 | 0.43
|  | 4$\times$ |  | 0.011 | 1.80 | 1.27 | 1.17 | 0.73 | 0.42
|  | 2$\times$ |  | 0.011 | 1.73 | 2.74 | 1.21 | 0.75 | 0.38
|  | 1$\times$ |  | 0.009 | 1.61 | 2.99 | 1.27 | 0.76 | 0.36
|  | 0.5$\times$ |  | 0.009 | 1.53 | 4.13 | 1.34 | 0.78 | 0.30
|  | 0.125$\times$ |  | 0.007 | 1.50 | 4.48 | 1.42 | 0.80 | 0.30
c) HyperSound – SIREN |  | $\bm{\omega_{0}}$ | $\bm{\omega_{i}}$ |  |  |  |  |  |
|  | 30 | 30 | 0.020 | 4.14 | -4.82 | 1.18 | 0.66 | 0.39
|  | 100 | 30 | 0.018 | 3.56 | -3.66 | 1.19 | 0.65 | 0.38
|  | 500 | 30 | 0.033 | 2.52 | -17.59 | 1.18 | 0.65 | 0.31
|  | 1000 | 30 | 0.036 | 2.50 | -22.63 | 1.19 | 0.66 | 0.32
|  | 2000 | 30 | 0.037 | 2.59 | -23.93 | 1.19 | 0.67 | 0.32
|  | 2000 | 100 | 0.036 | 1.63 | -22.11 | 1.18 | 0.67 | 0.36
d) Individual INR training |  | Model |  |  |  |  |  |  |
(upper bound) |  | FMLP |  | 0.001 | 1.39 | 17.19 | 2.31 | 0.95 | 0.23
|  | SIREN |  | <0.001 | 1.40 | 27.53 | 3.05 | 0.97 | 0.28

###  4.1 Setting

For most experiments, unless explicitly stated otherwise, we use the VCTK
dataset downsampled to $f=$22\,050\text{\,}\mathrm{H}\mathrm{z}$$, with
recording length set to $32\,768$ samples, which is around $1.5$ seconds. We
obtain inputs by cropping the original audio to the desired length. We use the
recordings of the last $10$ speakers in the VCTK dataset as a validation set.
We set batch size to for all the models to $16$, and train each model for
$2500$ epochs with $10\,000$ training samples per epoch. We use a OneCycleLR
(Smith & Topin, 2019) optimizer, varying learning rate depending on the target
network model type, as we observe that SIREN networks require smaller learning
rates to maintain stability during training.

During the evaluation, we focus on the quality of reconstructions obtained
with target networks. Example of these reconstructions can be seen in Figure
3. Since there is no consensus on a single approach for the quantitative
evaluation of audio quality, we assess the reconstruction results with
multiple metrics. We use quantitative metrics such as MSE, Log-Spectral
Distance (LSD) (Liu et al., 2022) and SI-SNR (Luo & Mesgarani, 2018). However,
since these metrics do not fully capture the perceptual quality of the audio
signal, we also include metrics such as PESQ (Rix et al., 2001), STOI (Taal et
al., 2010) and CDPAM (Manocha et al., 2021) to approximately measure how well
the reconstructed signal sounds to the human ear.

Unless explicitly mentioned otherwise, all the network hyperparameters are set
as described in Section 2. For training all of the hypernetworks, we use the
same data augmentations as in RAVE(Caillon & Esling, 2021), i.e. random crop,
phase mangle, and dequantization.

###  4.2 Reconstruction quality with FMLP target network

We measure the quality of reconstructions obtained on the VCTK dataset with
HyperSound using different variants of the target network. We compare the
quality of reconstructions with the state-of-the-art RAVE (Caillon & Esling,
2021) autoencoder retrained with sampling rate of $22\,050$ Hz. We use the
same set of data augmentations for training both RAVE and HyperSound, and
train RAVE for $3\text{\times}{10}^{6}$ steps.

We consider multiple variants of target networks that match the desired
compression ratio (measured as a ratio between the number of samples in the
input recording and the number of target network parameters). We set the
hypernetwork learning rate to $10^{-4}$ and use the AdamW optimizer. All
target networks use five hidden dense layers with biases, and we set the width
of each hidden layer to an equal value, so that we obtain the desired
compression ratio. We explore more realistic scenarios with compression ratios
above 1$\times$, but also include the results for target networks with higher
parameter counts as an insight into the representational capacities of over-
parameterized networks.

We show the results for RAVE in Table 2a, and for our method with FMLP target
network Table 2b. The results show that, while over-parameterization of the
target network is beneficial for the reconstruction quality, the experiments
with smaller target networks still obtain passable scores across all metrics.
We also notice that our method outperforms RAVE in pure reconstruction metrics
across all selected target network sizes, but obtains worse scores on metrics
that measure spectral distance and perceptual quality.

###  4.3 Training SIREN target networks

Despite (Benbarka et al., 2022) showing a close relationship between SIREN and
FMLP INRs, we observe that they behave very differently during training when
compared to FMLP networks. First, we notice that without the hand-crafted
hypernetwork weight initialization, as described in Section 3.3, SIREN
networks fail to learn any meaningful signal. Additionally, we notice that
SIRENs require significantly slower learning rates to converge, and we use a
learning rate of $10^{-6}$ ($\nicefrac{{1}}{{100}}$ the learning rate we use
in FMLP networks). We would like to use high values of $\omega_{0}$ and
$\omega_{i}$, which improve the network’s capability to represent high
frequencies, but increasing those values makes training the network very
unstable and often prevents convergence to any meaningful solution altogether.

Table 2c shows the results obtained for SIREN networks with different values
of $\omega_{0}$ and $\omega_{i}$. We use network architecture corresponding to
the FMLP model with a compression ratio of $1$$\times$. Increasing the
frequencies $\omega_{0}$ and $\omega_{i}$ slightly improves the perceptual
quality of the reconstructions, but makes the training process less stable. In
further experiments, we focus on FMLP networks, as they produce better results
and are easier to train.

Table 3: Comparison of different methods of generating INRs with hypernetworks described in Section 3.5. We notice that sharing initial layers of the target network is beneficial to reconstruction quality while making the effective size of both hypernetwork and target network smaller. model type | MSE | LSD | SI-SNR
---|---|---|---
standard | 0.009 | 1.61 | 2.99
multiplicative | 0.009 | 1.61 | 2.77
additive | 0.008 | 1.52 | 2.98
first layer shared | 0.008 | 1.58 | 3.67
first two layers shared | 0.008 | 1.60 | 3.44

###  4.4 Upper bound of the INR reconstruction quality

We compute our metrics for SIREN and FMLP INRs trained in vanilla fashion to
get an upper bound for reconstruction error. We select a subset of the
validation data, train a model for each example minimizing the mean squared
reconstruction error and report the mean metric values as an upper bound for
HyperSound. We only use a subset of the validation dataset, as the mean of the
metrics quickly stabilises after several recordings. We use basic variants of
the FMLP and SIREN networks ($\bm{\omega_{0}}=2000$, $\bm{\omega_{i}}=30$),
with parameters other than layer weights and biases set at fixed values as
described in Section 3.1 and the basic model size as described in previous
Sections.

The comparison of our results is described in Table 2d. Unsurprisingly,
vanilla INRs achieve significantly better quality compared to our method and
RAVE. Contrary to results for HyperSound, SIREN INRs seem to work better than
FMLP models when optimized directly for a single signal. We hypothesise that
this can be explained by the increased complexity of hypernetwork optimization
and ReLU nonlinearity being more robust to weight perturbations.

###  4.5 Exploring different ways to use weights generated by the hypernetwork

We explore possible extensions of the hypernetwork framework for INR
generation, comparing three approaches described in Section 3.5 with the
standard variant of the target network. We conduct experiments using the
network architecture corresponding to the FMLP with compression ratio
$1$$\times$ from Section 4.2.

The results of our experiments are described in Table 3. We notice that
learning a shared set of weights for the first layer in the target network
leads to better results than in the case of the standard model, but sharing
more than one layer is not beneficial. Sharing weights of the first layer can
be thought of as learning a shared feature extractor for smaller hypernetwork
and target network models, which means that effectively we reduce the
computational cost of the method and provide more compressed representations.
Other variants such as multiplicative and additive do not improve the results
noticeably while inducing additional computational cost due to the added
complexity.

###  4.6 Higher-frequency weighting in the loss function

We notice that our solution tends to produce representations that focus on
lower frequencies, which lowers the perceptual quality of the reconstructions.
We aim to improve this quality by assigning higher weights to the higher
frequencies in the STFT loss, as described in Section 3.4. As shown in Table
4, this simple modification improves the reconstruction quality of the model,
which indicates that audio processing can greatly benefit from domain-specific
loss functions.

Table 4: Impact of the frequency weighting on target network performance. A higher value of $p$ indicates a stronger bias towards high frequencies. $p=0$ equals no weighting. Our results indicate that focusing on high-frequency content in signals leads to better reconstructions. p | MSE | LSD | SI-SNR
---|---|---|---
0.0 [baseline] | 0.009 | 1.61 | 2.99
0.2 | 0.008 | 1.50 | 3.38
0.5 | 0.008 | 1.57 | 3.09
1.0 | 0.007 | 1.48 | 3.60

###  4.7 Results on other datasets

Table 5: Comparison of HyperSound reconstruction quality on different datasets. Our method still performs reasonably well on datasets other than VCTK, without any additional hyperparameter tuning. Dataset | MSE | LSD | SI-SNR
---|---|---|---
VCTK | 0.009 | 1.61 | 2.99
LJ Speech | 0.014 | 1.65 | -1.08
LibriSpeech | 0.011 | 2.15 | 0.89

In our experiments, we focus on the VCTK dataset, as most publicly available
audio datasets are either small, include data that does not match our sampling
rate or input length requirements, or only include data from a single source
or speaker. However, training hypernetworks requires large and diverse
datasets and benefits from data augmentations, which are harder to apply in a
reasonable way to music or environmental sound datasets. Therefore, we
evaluate HyperSound reconstruction quality on additional speech datasets,
i.e., LJ Speech and LibriSpeech train-clean-360 datasets. We present the
results of this evaluation in Table 5, showing that HyperSound still achieves
reasonable results despite no additional dataset-specific hyperparameter
tuning apart from learning rate reduction for LibriSpeech.

###  4.8 Representational capability of INRs

Finally, we conduct a simple experiment visualizing the weights of the INRs
obtained on the validation set of VCTK with t-SNE (Van der Maaten & Hinton,
2008) and show the results in Figure 4. We group the INRs by speaker identity,
as its the best-defined criterion to distinguish between recordings in the
dataset. We notice that different speakers occupy distinct regions in the
t-SNE subspace, with a clear separation between recordings of male and female
speech. This proves that representations produced by HyperSound capture the
underlying structure of the data.

##  5 Conclusions

We propose the first method of obtaining generalizable Implicit Neural
Representations for audio signals, named HyperSound. We adapt the hypernetwork
framework to audio processing through domain-specific modifications and
demonstrate that our model performs reasonably well on the VCTK dataset,
producing reconstructions quantitatively comparable to the state-of-the-art
RAVE autoencoder. We investigate the use of SIREN and Fourier-mapped
perceptron networks as INRs generated by the hypernetwork and try alternative
ways of applying hypernetworks in the INR generation, which improve upon the
standard method. INRs produced by our method can represent the signal in
compressed way, using less weights than the number of samples in the original
recording. Additionally, we demonstrate that our method performs reasonably
across different datasets with minimal hyperparameter tuning and that the
representations produced by the method align with the underlying structure of
the data.

Figure 4: t-SNE visualisation of the INR weights obtained on VCTK validation
set. Different colors of the dots indicate different speakers. Speakers are
generally clustered, and we notice a clear separation of the male and female
recordings.

Limitations and future work. As theoretical guarantees of hypernetwork
training are hard to define, we find our method sensitive to the
hyperparameter choice and its training requires a lot of trial and error.
Moreover, the misalignment between the human perception of the sound and
metrics optimized in deep neural networks means that the perceptual quality of
reconstructions generated with our method leaves room for improvement. We hope
that further work in those directions can improve the stability of our method
and better calibrate the results to human perception.

This work was supported by Foundation for Polish Science (grant no
POIR.04.04.00-00-14DE/18-00) carried out within the Team-Net program co-
financed by the European Union under the European Regional Development Fund,
as well as the National Centre of Science (Poland) Grant No.
2020/39/B/ST6/01511. Przemysław Spurek is supported by the National Centre of
Science (Poland) Grant No. 2021/43/B/ST6/01456.

## References

  * Benbarka et al. (2022) Benbarka, N., Höfer, T., Riaz, H. u.-M., and Zell, A.  Seeing Implicit Neural Representations As Fourier Series.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , pp. 2041–2050, January 2022.
  * Caillon & Esling (2021) Caillon, A. and Esling, P.  RAVE: A variational autoencoder for fast and high-quality neural audio synthesis.  _arXiv preprint arXiv:2111.05011_ , 2021.
  * Chan et al. (2021) Chan, E. R., Monteiro, M., Kellnhofer, P., Wu, J., and Wetzstein, G.  PI-GAN: Periodic implicit generative adversarial networks for 3d-aware image synthesis.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 5799–5809, 2021.
  * Chang et al. (2019) Chang, O., Flokas, L., and Lipson, H.  Principled weight initialization for hypernetworks.  In _International Conference on Learning Representations_ , 2019.
  * Chen & Wang (2022) Chen, Y. and Wang, X.  Transformers as meta-learners for implicit neural representations.  In _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVII_ , pp. 170–187. Springer, 2022.
  * Clevert et al. (2015) Clevert, D.-A., Unterthiner, T., and Hochreiter, S.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs).  _arXiv preprint arXiv:1511.07289_ , 2015.
  * Défossez et al. (2018) Défossez, A., Zeghidour, N., Usunier, N., Bottou, L., and Bach, F.  SING: Symbol-to-Instrument Neural Generator.  _Advances in neural information processing systems_ , 31, 2018.
  * Dupont et al. (2022) Dupont, E., Kim, H., Eslami, S., Rezende, D., and Rosenbaum, D.  From data to functa: Your data point is a function and you should treat it like one.  _arXiv preprint arXiv:2201.12204_ , 2022.
  * Engel et al. (2017) Engel, J., Resnick, C., Roberts, A., Dieleman, S., Norouzi, M., Eck, D., and Simonyan, K.  Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders.  In _International Conference on Machine Learning_ , pp. 1068–1077. PMLR, 2017.
  * Fons et al. (2022) Fons, E., Sztrajman, A., El-Laham, Y., Iosifidis, A., and Vyetrenko, S.  HyperTime: Implicit Neural Representation for Time Series.  _arXiv preprint arXiv:2208.05836_ , 2022.
  * Glorot & Bengio (2010) Glorot, X. and Bengio, Y.  Understanding the difficulty of training deep feedforward neural networks.  In _Proceedings of the thirteenth international conference on artificial intelligence and statistics_ , pp. 249–256. JMLR Workshop and Conference Proceedings, 2010.
  * Ha et al. (2016) Ha, D., Dai, A., and Le, Q. V.  HyperNetworks.  _arXiv preprint arXiv:1609.09106_ , 2016.
  * He et al. (2015) He, K., Zhang, X., Ren, S., and Sun, J.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 1026–1034, 2015.
  * Karras et al. (2018) Karras, T., Aila, T., Laine, S., and Lehtinen, J.  Progressive Growing of GANs for Improved Quality, Stability, and Variation.  In _International Conference on Learning Representations_ , 2018.
  * Klocek et al. (2019) Klocek, S., Maziarka, Ł., Wołczyk, M., Tabor, J., Nowak, J., and Śmieja, M.  Hypernetwork functional image representation.  In _International Conference on Artificial Neural Networks_ , pp. 496–510. Springer, 2019.
  * Kumar et al. (2019) Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W. Z., Sotelo, J., de Brébisson, A., Bengio, Y., and Courville, A. C.  MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis.  _Advances in neural information processing systems_ , 32, 2019.
  * Liu et al. (2022) Liu, H., Choi, W., Liu, X., Kong, Q., Tian, Q., and Wang, D.  Neural Vocoder is All You Need for Speech Super-resolution.  _arXiv preprint arXiv:2203.14941_ , 2022.
  * Luo & Mesgarani (2018) Luo, Y. and Mesgarani, N.  TasNet: time-domain audio separation network for real-time, single-channel speech separation.  In _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 696–700. IEEE, 2018.
  * Manocha et al. (2021) Manocha, P., Jin, Z., Zhang, R., and Finkelstein, A.  CDPAM: Contrastive learning for perceptual audio similarity.  In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 196–200. IEEE, 2021.
  * Mehri et al. (2016) Mehri, S., Kumar, K., Gulrajani, I., Kumar, R., Jain, S., Sotelo, J., Courville, A., and Bengio, Y.  SampleRNN: An Unconditional End-to-End Neural Audio Generation Model.  _arXiv preprint arXiv:1612.07837_ , 2016.
  * Mehta et al. (2021) Mehta, I., Gharbi, M., Barnes, C., Shechtman, E., Ramamoorthi, R., and Chandraker, M.  Modulated Periodic Activations for Generalizable Local Functional Representations.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 14214–14223, 2021.
  * Mildenhall et al. (2021) Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R.  NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.  _Communications of the ACM_ , 65(1):99–106, 2021\.
  * Müller et al. (2022) Müller, N., Siddiqui, Y., Porzi, L., Bulò, S. R., Kontschieder, P., and Nießner, M.  Diffrf: Rendering-guided 3d radiance field diffusion.  _arXiv preprint arXiv:2212.01206_ , 2022.
  * Oord et al. (2018) Oord, A., Li, Y., Babuschkin, I., Simonyan, K., Vinyals, O., Kavukcuoglu, K., Driessche, G., Lockhart, E., Cobo, L., Stimberg, F., et al.  Parallel WaveNet: Fast High-Fidelity Speech Synthesis.  In _International conference on machine learning_ , pp. 3918–3926. PMLR, 2018.
  * Oord et al. (2016) Oord, A. v. d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K.  WaveNet: A Generative Model for Raw Audio.  _arXiv preprint arXiv:1609.03499_ , 2016.
  * Purwins et al. (2019) Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S.-Y., and Sainath, T.  Deep learning for audio signal processing.  _IEEE Journal of Selected Topics in Signal Processing_ , 13(2):206–219, 2019.
  * Rahaman et al. (2019) Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., and Courville, A.  On the spectral bias of neural networks.  In _International Conference on Machine Learning_ , pp. 5301–5310. PMLR, 2019.
  * Rix et al. (2001) Rix, A. W., Beerends, J. G., Hollier, M. P., and Hekstra, A. P.  Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs.  In _2001 IEEE international conference on acoustics, speech, and signal processing. Proceedings (Cat. No. 01CH37221)_ , volume 2, pp. 749–752. IEEE, 2001.
  * Sitzmann et al. (2020a) Sitzmann, V., Chan, E., Tucker, R., Snavely, N., and Wetzstein, G.  Metasdf: Meta-learning signed distance functions.  _Advances in Neural Information Processing Systems_ , 33:10136–10147, 2020a.
  * Sitzmann et al. (2020b) Sitzmann, V., Martel, J., Bergman, A., Lindell, D., and Wetzstein, G.  Implicit Neural Representations with Periodic Activation Functions.  _Advances in Neural Information Processing Systems_ , 33:7462–7473, 2020b.
  * Smith & Topin (2019) Smith, L. N. and Topin, N.  Super-convergence: Very fast training of neural networks using large learning rates.  In _Artificial intelligence and machine learning for multi-domain operations applications_ , volume 11006, pp. 369–386. SPIE, 2019\.
  * Spurek et al. (2020) Spurek, P., Winczowski, S., Tabor, J., Zamorski, M., Zięba, M., and Trzciński, T.  Hypernetwork approach to generating point clouds.  _arXiv preprint arXiv:2003.00802_ , 2020.
  * Strümpler et al. (2022) Strümpler, Y., Postels, J., Yang, R., Gool, L. V., and Tombari, F.  Implicit neural representations for image compression.  In _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVI_ , pp. 74–91. Springer, 2022.
  * Taal et al. (2010) Taal, C. H., Hendriks, R. C., Heusdens, R., and Jensen, J.  A short-time objective intelligibility measure for time-frequency weighted noisy speech.  In _2010 IEEE international conference on acoustics, speech and signal processing_ , pp. 4214–4217. IEEE, 2010.
  * Tancik et al. (2020) Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., and Ng, R.  Fourier features let networks learn high frequency functions in low dimensional domains.  _Advances in Neural Information Processing Systems_ , 33:7537–7547, 2020.
  * Van der Maaten & Hinton (2008) Van der Maaten, L. and Hinton, G.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(11), 2008.
  * Von Oswald et al. (2019) Von Oswald, J., Henning, C., Sacramento, J., and Grewe, B. F.  Continual learning with hypernetworks.  _arXiv preprint arXiv:1906.00695_ , 2019.
  * Yamamoto et al. (2020) Yamamoto, R., Song, E., and Kim, J.-M.  Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram.  In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 6199–6203. IEEE, 2020.
  * Zeghidour et al. (2021) Zeghidour, N., Luebs, A., Omran, A., Skoglund, J., and Tagliasacchi, M.  SoundStream: An End-to-End Neural Audio Codec.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 30:495–507, 2021.
  * Zhao et al. (2020) Zhao, D., von Oswald, J., Kobayashi, S., Sacramento, J., and Grewe, B. F.  Meta-Learning via Hypernetworks.  2020\.
  * Zuiderveld et al. (2021) Zuiderveld, J., Federici, M., and Bekkers, E. J.  Towards Lightweight Controllable Audio Synthesis with Conditional Implicit Neural Representations.  _arXiv preprint arXiv:2111.08462_ , 2021.