# MoreauPruner: Robust Pruning of Large Language Models against Weight
Perturbations

Zixiao Wang, Jingwei Zhang, Wenqian Zhao, Farzan Farnia, Bei Yu,
The Chinese University of Hong Kong
{zxwang22, jwzhang22, wqzhao, farnia<EMAIL_ADDRESS>

###### Abstract

Few-shot gradient methods have been extensively utilized in existing model
pruning methods, where the model weights are regarded as static values and the
effects of potential weight perturbations are not considered. However, the
widely used large language models (LLMs) have several billion model
parameters, which could increase the fragility of few-shot gradient pruning.
In this work, we experimentally show that one-shot gradient pruning algorithms
could lead to unstable results under perturbations to model weights. And the
minor error of switching between data formats bfloat16 and float16 could
result in drastically different outcomes. To address such instabilities, we
leverage optimization analysis and propose an LLM structural pruning method,
called MoreauPruner, with provable robustness against weight perturbations. In
MoreauPruner, the model weight importance is estimated based on the neural
network’s Moreau envelope, which can be flexibly combined with $\ell_{1}$-norm
regularization techniques to induce the sparsity required in the pruning task.
We extensively evaluate the MoreauPruner algorithm on several well-known LLMs,
including LLaMA-7B, LLaMA-13B, LLaMA3-8B, and Vicuna-7B. Our numerical results
suggest the robustness of MoreauPruner against weight perturbations, and
indicate the MoreauPruner’s successful accuracy-based scores in comparison to
several existing pruning methods. We have released the code in
https://github.com/ShiningSord/MoreauPruner.

##  1 Introduction

In the rapidly evolving field of Natural Language Processing (NLP),
transformer-based Large Language Models (LLMs) such as GPTs [1] and LLaMAs [2,
3] have become foundational technologies, driving significant advances across
various tasks. These models excel in understanding and generating human
language due to their scalable architecture, which allows performance to
improve with an increase in parameters. However, deploying these large models
poses significant challenges due to their substantial computational and memory
demands. To address these challenges, considerable research has been directed
toward model pruning [4, 5, 6, 7], a technique aimed at reducing model size
while maintaining or enhancing model performance.

While effective in accelerating LLMs for efficient deployment, existing
pruning methods generally treat the parameters of a pre-trained language model
as fixed, neglecting potential perturbations in the weights. These
perturbations can originate from various sources, including quantization
errors during transitions between precision levels and alterations from
parameter-efficient fine-tuning (PEFT) technologies that modify a small subset
of parameters. When there is a need to prune those slightly modified models,
we may expect that the pruning results getting from modified models similar to
those of the original models. For example, existing LLMs are usually trained
with the weight format bfloat16 (BF16) and deployed with the weight format
float16 (FP16). As both BF16 and FP16 utilize 16-bit to represent a floating
point, the negligible transition error will not affect inference results in
most cases. Considering that the basic idea of pruning is removing unnecessary
weights and keeping the essential weights, it is straightforward to believe
that the models pruned from BF16 and FP16 will be close to each other.
However, current gradient-dependent pruning methods [6, 7, 8, 9] utilize
gradient to indicate the importance of weight elements while gradient is known
to be sensitive to such weight perturbations, leading to significant
variations in pruning outcomes, as depicted in Figure 1. Such inconsistency in
pruned outcomes could be anti-intuitive, and a robust pruning algorithm
against weight perturbation may further enhance the performance of the
compressed models.

Figure 1: While gradient-based pruning methods are sensitive to weight
perturbation, the proposed MoreauPruner gives a robust estimation of weight
importance.

This paper introduces MoreauPruner, a novel robust structural pruning
algorithm for LLMs, designed to mitigate the effects of weight perturbations
while preserving model performance. MoreauPruner utilizes the gradient of the
loss function’s Moreau envelope [10, 11, 12], a well-established optimization
tool for function smoothing, to reduce weight sensitivity to perturbations
during the pruning process. We show that the gradient of the Moreau envelope
remains stable within the neighborhood of given weight in parameter space.
This stability enables MoreauPruner to generate robustness pruning result
against weight perturbations, with any norm-bounded perturbation resulting in
only a bounded change of the gradient. Additionally, by incorporating an
$\ell_{1}$-group-norm-based regularization penalty, MoreauPruner promotes
group-level sparsity in the gradient, which is suitable for structural pruning
to facilitate real-life acceleration on hardware platforms. Our empirical
results suggest that MoreauPruner improves the robustness of pruning outcomes
against weight perturbations and achieves state-of-the-art post-pruning model
performance among baseline LLM pruning methods.

Our contributions through this work are threefold:

  * •

We emphasize the importance of considering weight perturbations in pruning
algorithms, an aspect previously neglected in the literature. This work is
among the first to tackle the robustness of pruning algorithms to such
perturbations.

  * •

We introduce MoreauPruner, a structural pruning algorithm that offers provable
robustness to weight perturbations, leveraging the Moreau envelope to ensure
the smoothness and stability of the pruning process.

  * •

Through extensive experimentation with widely-used large language models such
as LLaMA-7B, LLaMA-13B, LLaMA3-8B, and Vicuna-7B, we demonstrate that
MoreauPruner achieves a state-of-the-art performance in both robustness to
weight perturbation and overall performance of compressed models.

##  2 Related Work

###  2.1 Efficient Large Language Models

Large Language Models (LLMs) [2, 13, 14, 3] have achieved remarkable
performance by following the scaling laws [15]. However, deploying LLMs can be
challenging due to high inference costs in resource-limited scenarios. Various
methods have been proposed to reduce model size, including knowledge
distillation [16, 17, 18, 19], which involves transferring the knowledge from
the original model to a smaller one; model quantization [1, 20, 21, 22], which
reduces the bit length of the model weights; and model pruning [4, 23, 24,
25], which involves removing non-essential weights to speed up inference. This
work primarily focuses on pruning LLMs [26, 6, 27, 28, 29], where gradients
are particularly sensitive to weight perturbations due to the large scale of
the model.

###  2.2 Pruning Criteria

To determine which weights to prune during the pruning phase, the importance
of each weight is assessed using various criteria. Several studies [30, 31, 4,
32] adopt a magnitude-based criterion, retaining weights with larger
magnitudes post-pruning. Recent approaches [30] also consider activation
values to evaluate weight importance. Some prevalent criteria are based on
Taylor Expansion approximation [8, 6, 33, 34, 9], utilizing differential
information (zero-th, first, and second order) to estimate output changes if
weights are removed. Notably, [7] highlights that, in LLMs, gradients can be
efficiently approximated using low-rank methods [35] when the direct
computation of the gradient is too costly. Nevertheless, gradients can be
highly sensitive to weight modifications, rendering gradient-based pruning
criteria susceptible to variations in weight. In response, MoreauPruner offers
proven robustness against any norm-bounded weight perturbation, maintaining
model performance.

##  3 Preliminaries

In this section, we provide a review of prior Taylor-expansion-based
structural pruning methods, along with the notation and definitions used in
the paper.

###  3.1 Notation and Definitions

Let $\mathcal{D}=\\{{\bm{x}}_{i}\\}_{i=1}^{N}$ denotes a text dataset with N
samples. $f({\bm{w}},{\bm{x}})$ is the next token prediction loss on sample
${\bm{x}}$ with a parameterized language model and its weight
${\bm{w}}\in\mathbb{R}^{d}$. Then the expectation of loss on dataset
$\mathcal{D}$ is

$L({\bm{w}},\mathcal{D})=\frac{1}{N}\sum_{i=1}^{N}f({\bm{w}},{\bm{x}}_{i}).$
(1)

The $\ell_{p}$-norm of an input vector ${\mathbf{v}}$ is represented as
$\|{\mathbf{v}}\|_{p}$. Furthermore, we use notation $\|{\mathbf{v}}\|_{p,q}$
to denote the $\ell_{p,q}$-group-norm of ${\mathbf{v}}$ defined in the
following equation for given variable subsets
$S_{1},...,S_{t}\subset\\{1,...,d\\}$:

$\|{\mathbf{v}}\|_{p,q}=\bigl{\|}\left[\|{\mathbf{v}}_{S_{1}}\|_{p},...,\|{\mathbf{v}}_{S_{t}}\|_{p}\right]\bigr{\|}_{q},$
(2)

which means $\|{\mathbf{v}}\|_{p,q}$ is the $\ell_{q}$-norm of a vector
containing the $\ell_{p}$-norms of the subvectors of ${\mathbf{v}}$
characterized by index subsets $S_{1},...,S_{t}\subset\\{1,...,d\\}$.

###  3.2 Estimating Importance Score via Taylor Expansion

Recent pruning method usually estimate the impact of removing a weight element
$w^{(k)}$ via Taylor Expansion,

$\displaystyle I(w^{(k)})$
$\displaystyle=\|L({\bm{w}},\mathcal{D})-L(w^{(k)}=0,\mathcal{D})\|$
$\displaystyle=|\frac{\partial L({\bm{w}},\mathcal{D})}{\partial
w^{(k)}}w^{(k)}-\frac{1}{2}w^{(k)}{\mathbf{H}}_{kk}({\mathbf{x}})w^{(k)}+{\mathcal{O}}(|w^{(k)}|^{3})|.$

In the above equations, $L(w^{(k)}=0,\mathcal{D})$ denotes masking out the
parameter ${\bm{w}}$ in the neural network. Hessian matrix ${\mathbf{H}}$ is
approximated by a diagonal one and ${\mathbf{H}}_{kk}$ is the $k$-th element
on the diagonal. In some of previous pruning methods, the first-term is
typically neglected since the the model is well-trained and converged on the
training dataset, where $\frac{\partial L({\bm{w}},\mathcal{D})}{\partial
w^{(k)}}\approx 0$. However, a recent LLM pruning work[6] point out the
calibration dataset used in pruning is out of the training data and
$\frac{\partial L({\bm{w}},\mathcal{D})}{\partial w^{(k)}}\neq 0$. Given that
the heavy computational cost of the Hessian matrix is unacceptable for LLMs,
unlike small models, the importance score of parameter $w^{(k)}$ can be
approximated using the first term in the Taylor Expansion,

$I(w^{(k)})=|\frac{\partial L({\bm{w}},\mathcal{D})}{\partial
w^{(k)}}w^{(k)}|.$ (3)

To achieve real-time acceleration on hardware, structural pruning algorithms
remove weight elements in group, i.e., all elements in a channel, blocks or
heads. The importance score of such a structure can be easy obtained by
summarizing the importance of elements,

$I({\bm{w}}_{i})=\sum_{k}I({\bm{w}}_{i}^{(k)}),$ (4)

where ${\bm{w}}_{i}$ is the weight vector in a structure.

Existing pruning algorithms tend to pruning those weights with smaller
importance scores. However, as we mentioned in Section 1, the simple gradient
in Equation 3 is sensitive to weight perturbations, which further leads to an
unstable pruning result. Motivated by this fact, we have designed a robust
pruning criterion in MoreauPruner and detailed it in the next section.

###  3.3 Dependency-aware Structural Pruning

Previous works[24, 6] suggest that structural pruning should consider
dependency among structures. Here, a weight group
${\mathcal{G}}=\\{{\bm{w}}_{i}\\}^{M}_{i=1}$ represents a collection of
coupled structures, where $M$ is the number of structures in one group, and
${\bm{w}}_{i}$ denotes the weight for each structure. The group can be
effiently detected by [24]. And the importance score of the group
${\mathcal{G}}$ is then estimated as follows:

$I({\mathcal{G}})=\mathop{Agg}\limits_{i=1}^{M}I({\bm{w}}_{i}),$ (5)

where $\mathop{Agg}$ is a customized aggregation function chosen from options
like Summation, Production, Max, etc. After assessing the importance of each
group, groups with lower importance are pruned to achieve a pre-determined
pruning ratio.

##  4 MoreauPruner

In this section, we introduce the proposed pruning method, MoreauPruner. We
start by detailing the proposed perturbation-robust pruning criteria. In the
second subsection, we introduce the two versions of MoreauPruner and how they
work.

###  4.1 Robustifying Gradient via Moreau Envelope

Here we leverage the notion of Moreau envelope from the convex optimization
literature to propose an optimization-based approach to robust gradient-based
pruning. The considered robust gradient follows Moreau-Yosida regularization,
based on which the Moreau envelope of a neural network’s parameters is defined
as follows.

######  Definition 1.

Consider function $g:\mathbb{R}^{d}\rightarrow\mathbb{R}$ and regularization
parameter $\rho>0$. The Moreau envelope of $g$ at input weight ${\bm{w}}$,
$g^{\rho}:\mathbb{R}^{d}\rightarrow\mathbb{R}$ is defined to be

$g^{\rho}({\bm{w}})\,:=\,\inf_{\tilde{{\bm{w}}}}g^{\rho}(\tilde{{\bm{w}}})+\frac{1}{2\rho}\bigl{\|}\tilde{{\bm{w}}}-{\bm{w}}\bigr{\|}^{2}_{2}.$
(6)

Following [11], we utilize the gradient of the Moreau envelope as a robust
evaluation of the local sensitivity of the loss function sensitivity to
altering the model weights.

######  Definition 2.

Given the input weight ${\bm{w}}$ and regularization parameter $\rho>0$, we
define MoreauGrad as the Moreau envelope $g^{\rho}$’s gradient
$\mathrm{MG}^{\rho}[g]:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$:

$\mathrm{MG}^{\rho}[g]({\bm{w}}):=\nabla g^{\rho}({\bm{w}}).$ (7)

To analyze the gradient of Moreau envelope, we first discuss the optimization-
based smoothing enforced by the Moreau envelope. Note that the Moreau envelope
is known as an optimization tool to turn non-smooth convex functions (e.g.
$\ell_{1}$-norm) into smooth functions, where the smoothness is usually
regarding the input variable ${\bm{x}}$. Here in the pruning case, we discuss
the smoothness regarding the function parameters ${\bm{w}}$ and extend the
result to the weakly-convex functions which also apply to non-convex
functions.

######  Theorem 1.

Suppose that the parameterized function
$g({\bm{w}}):\mathbb{R}^{d}\rightarrow\mathbb{R}$ is $\beta$-Lipschitz, i.e.
it satisfies
$\bigl{|}g({\bm{w}})-g(\mathbf{v})\bigr{|}\leq\beta\|{\bm{w}}-\mathbf{v}\|_{2}$
for every $\mathbf{v,w}\in\mathbb{R}^{d}$. Consider the Gaussian-smoothed
$g_{\sigma}(\mathbf{w})=\mathbb{E}_{\mathbf{u}\sim\mathcal{N}(\mathbf{0},\sigma^{2}I)}\bigl{[}g({\bm{w}}+\mathbf{u})\bigr{]}$.
Then, for every $0<\rho<\frac{\sigma}{\beta}$, the following robustness
guarantee will hold of the Moreau envelope of the Gaussian-smoothed
$g_{\sigma}^{\rho}({\bm{w}})$

$\|\mathrm{MG}^{\rho}[g_{\sigma}]({\bm{w}}_{1})-\mathrm{MG}^{\rho}[g_{\sigma}]({\bm{w}}_{2})\|_{2}\leq\frac{\sigma}{\min\\{\sigma\rho,\sigma-\rho\beta\\}}\|{\bm{w}}_{1}-{\bm{w}}_{2}\|_{2}.$

###### Proof.

We defer the proof of the above theorem into appendices due to the space
limitation. ∎

We note that, as shown in [11], the above definition can be combined with
sparsity-based norm penalties, such as $\ell_{1}$-norm $\|\cdot\|_{1}$ or
$\ell_{2,1}$-group norm $\|\cdot\|_{2,1}$ norm. Here, we generalize [11]’s
definition of (group)sparse-Moreau envelop, and given a convex function
$h:\mathbb{R}^{d}\rightarrow\mathbb{R}$ propose the following definition of
$h$-Moreau envelope:

######  Definition 3.

Given convex function $h$, input weight ${\bm{w}}$, and regularization
parameter $\rho>0$, we define $h$-MoreauGrad of function $g$, denoted by
$h\text{-}\mathrm{MG}^{\rho}[g](\mathbf{w})$, as
$\frac{1}{\rho}\bigl{(}\mathbf{v}^{*}(\mathbf{w})-\mathbf{w}\bigr{)}$ where
$\mathbf{v}^{*}(\mathbf{w})$ denotes the optimal solution to the following
optimization problem:

$\min_{\mathbf{v}\in\mathbb{R}^{d}}\>g(\mathbf{v})+\frac{1}{2\rho}\bigl{\|}\mathbf{v}-\mathbf{w}\bigr{\|}_{2}^{2}+h\bigl{(}\mathbf{v}-\mathbf{w}\bigr{)}.$
(8)

In our numerical analysis, we specifically focus on GroupSparse-MoreauGrad
which is the $h$-MoreauGrad with the group-norm
$h(\mathbf{v})=\eta\|\mathbf{v}\|_{2,1}$. $\eta$ is the sparsity parameter.
Here, we extend the robustness guarantee of Theorem 1 to a general
$h$-MoreauGrad.

######  Theorem 2.

Consider the setting of Theorem 1 and suppose $h$ is a convex function. Then,
for every $0<\rho<\frac{\sigma}{\beta}$, the following robustness guarantee
will hold of the $h$-Moreau envelope of $g_{\sigma}^{\rho}({\bm{w}})$

$\|h\text{-}\mathrm{MG}^{\rho}[g_{\sigma}]({\bm{w}}_{1})-h\text{-}\mathrm{MG}^{\rho}[g_{\sigma}]({\bm{w}}_{2})\|_{2}\leq\frac{\sigma}{\min\\{\sigma\rho,\sigma-\rho\beta\\}}\|{\bm{w}}_{1}-{\bm{w}}_{2}\|_{2}.$

###### Proof.

We defer the proof of the above theorem into appendices due to the space
limitation. ∎

Algorithm 1 MoreauPruner Algorithm

0: samples ${\bm{x}}$, network with parameter $f({\bm{w}})$, regularization
parameter $\rho$, group-sparsity $\eta$, noise std $\sigma$, stepsize
$\gamma$, optimization length $T$

1: Initialize ${\bm{w}}^{(0)}={\bm{w}}$

2: for $t=0,\ldots,T$ do

3: draw noise vectors
${\bm{z}}_{1},\ldots,{\bm{z}}_{m}\sim\mathcal{N}(0,\sigma^{2}I_{d\times d})$

4: compute ${\bm{g}}_{t}=\frac{1}{m}\sum_{i=1}^{m}\nabla
f({\bm{w}}^{(t)}+{\bm{z}}_{i},{\bm{x}})$

5: Update
${\bm{w}}^{(t+1)}\leftarrow(1-\frac{\gamma}{\rho}){\bm{w}}^{(t)}-\gamma({\bm{g}}_{t}-\frac{1}{\rho}{\bm{w}})$

6: if GroupSparse then

7: Update
${\bm{w}}^{(t+1)}\leftarrow\text{GroupSoftThreshold}_{\gamma\eta}({\bm{w}}^{(t+1)}-{\bm{w}})+{\bm{w}}$

8: end if

9: end for

10: Compute Importance score
$I({\bm{w}})=\frac{1}{\rho}({\bm{w}}^{(T)}-{\bm{w}}){\bm{w}}$

11: Prune network $f({\bm{w}})$ according to $I({\bm{w}})$

12: Finetune pruned network

13: Return finetuned network

###  4.2 Leveraging MoreauGrad for Robust Pruning

MoreauPruner. With the defined MoreauGrad, we established a robust estimation
on the influence of removing a weight element $w^{(k)}$ based on Equation 3,

$\mathrm{MG}^{\rho}\text{-}I(w^{(k)})=|\mathrm{MG}^{\rho}[g]({\bm{w}})\odot{\bm{w}}|^{(k)},$
(9)

where $g({\bm{w}})$ is the exception of loss function
$L({\bm{w}},\mathcal{D})$ defined in Equation 1, $|\cdot|$ is an element-wise
absolute value function and $\odot$ denotes Hadamard product. To compute, the
optimal solution $\tilde{{\bm{w}}}^{*}_{\rho}({\bm{w}})$ of the optimization
problem of Moreau Envelope $g^{\rho}({\bm{w}})$ can be obtained over a
calibration dataset with the first-order gradient descent optimization method.
Since the difference $\tilde{{\bm{w}}}^{*}_{\rho}({\bm{w}})-{\bm{w}}$ is
aligned with $g^{\rho}$’s gradient[10, 11], we have,

$\nabla
g^{\rho}({\bm{w}})=-\frac{1}{\rho}(\tilde{{\bm{w}}}^{*}_{\rho}({\bm{w}})-{\bm{w}}).$
(10)

Then, the robust importance score of the structure ${\bm{w}}_{i}$ and group
$\mathcal{G}$ can be also estimated by Equation 4 and Equation 5. We denote
the structural pruning method removing groups with smaller robust importance
score as MoreauPruner.

MoreauPruner-GS. Similar to MoreauPruner, the robust estimation on the
influence of removing parameter ${\bm{w}}^{k}$ is,

$h\text{-}\mathrm{MG}^{\rho}\text{-}I(w^{(k)})=|h\text{-}\mathrm{MG}^{\rho}[g_{\sigma}]({\bm{w}})\odot{\bm{w}}|^{(k)},$
(11)

where the group sparsity is conducted at the channel level in our
implementation, i.e., each variable subset in the $2,1$-group-norm is a
channel in the model. To compute the GroupSparse-MoreauGrad, we utilize the
proximal gradient descent algorithm as described in Algorithm 1. Note that we
apply the group-soft-thresholding function as the proximal operator for the
$\ell_{2,1}$-norm function present in GroupSparse-MoreauGrad,

$\mathrm{GST}_{\alpha}({\bm{v}})_{S_{i}}:=\begin{cases}0&\text{if
}\|{\bm{v}}_{S_{i}}\|_{2}\leq\alpha\\\
\left(1-\frac{\alpha}{\|{\bm{v}}_{S_{i}}\|_{2}}\right){\bm{v}}_{S_{i}}&\text{if
}\|{\bm{v}}_{S_{i}}\|_{2}>\alpha.\end{cases}$

Once the optimization of $h$-Moreau envelope ends, the GroupSparse-MoreauGrad
can be calculated according to Definition 3. We treat the method as
MoreauPruner-GS to mark the group sparsity of importance score obtained during
optimization.

##  5 Numerical Results

In this section, we conducted experiments on several famous LLMs to evaluate
the proposed MoreauPruner’s performance and support our theoretical claim. We
also provided further insights in the discussion subsection on how and why
MoreauPruner works well.

Table 1: Algorithms’ robustness against weight perturbation. Diff denotes the
absolute difference between weight formats, bfloat16 and float16. Rounding
results in changes to the last digit.

Method | Format | Pruning Ratio
---|---|---
5% | 10% | 15% | 20%
LLM-Pruner[6] | BF16 | 13.80 | 17.73 | 32.60 | 95.82
FP16 | 13.75 | 17.79 | 32.10 | 96.57
Diff($\downarrow$) | 0.05 | 0.07 | 0.51 | 0.75
MoreauPruner | BF16 | 13.89 | 17.42 | 31.05 | 91.79
FP16 | 13.83 | 17.45 | 30.99 | 91.79
Diff($\downarrow$) | 0.05 | 0.03 | 0.06 | 0.00

(a) WikiText2

Method | Format | Pruning Ratio
---|---|---
5% | 10% | 15% | 20%
LLM-Pruner[6] | BF16 | 25.00 | 32.10 | 61.87 | 202.86
FP16 | 24.85 | 32.16 | 61.15 | 210.12
Diff($\downarrow$) | 0.15 | 0.06 | 0.72 | 7.26
MoreauPruner | BF16 | 25.00 | 32.22 | 60.43 | 176.24
FP16 | 24.95 | 32.29 | 60.43 | 174.19
Diff($\downarrow$) | 0.05 | 0.06 | 0.00 | 2.05

(b) PTB

Table 2: Zero-shot performance of the Pruned LLaMA-7B. † denotes results from
[6], ‡ denotes results from [7] and ∗ is implemented according to open-source
code. The best results is bold. The methods proposed in this paper are filled
with green.

Pruning Ratio | Method | WikiText2($\downarrow$) | PTB($\downarrow$) | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average
---|---|---|---|---|---|---|---|---|---|---|---
Ratio = 0% | LLaMA-7B† | 12.62 | 22.14 | 73.18 | 78.35 | 72.99 | 67.01 | 67.45 | 41.38 | 42.40 | 63.25
Ratio = 20% w/o tune | Magnitude† | 582.41 | 1022.17 | 59.66 | 58.00 | 37.04 | 52.41 | 33.12 | 28.58 | 29.80 | 42.65
Random† | 27.51 | 43.19 | 61.83 | 71.33 | 56.26 | 54.46 | 57.07 | 32.85 | 35.00 | 52.69
WANDA‡[30] | 22.12 | 38.19 | 64.93 | 70.14 | 58.12 | 55.39 | 56.63 | 33.98 | 35.43 | 53.23
LLM-Pruner∗[6] | 19.09 | 34.23 | 56.91 | 75.08 | 66.81 | 60.06 | 60.94 | 36.43 | 40.00 | 56.60
LoRAPrune‡[7] | 20.67 | 34.12 | 57.98 | 75.11 | 65.81 | 59.90 | 62.14 | 34.59 | 39.98 | 56.50
SmoothGrad | 18.91 | 34.30 | 59.60 | 75.14 | 65.98 | 61.01 | 60.77 | 37.12 | 39.80 | 57.06
MoreauPruner | 18.61 | 32.92 | 55.44 | 76.17 | 66.47 | 63.61 | 61.53 | 37.80 | 40.60 | 57.37
MoreauPruner-GS | 18.72 | 34.91 | 62.51 | 75.52 | 68.29 | 62.75 | 54.88 | 36.35 | 40.80 | 57.30
Ratio = 20% w tune | WANDA‡[30] | 18.43 | 33.16 | 65.75 | 74.70 | 64.52 | 59.35 | 60.65 | 36.26 | 39.40 | 57.23
LLM-Pruner∗[6] | 17.62 | 30.57 | 65.78 | 76.44 | 68.67 | 64.33 | 63.26 | 36.35 | 41.00 | 59.40
LoRAPrune‡[7] | 16.80 | 28.75 | 65.62 | 79.31 | 70.00 | 62.76 | 65.87 | 37.69 | 39.14 | 60.05
SmoothGrad | 17.45 | 30.57 | 66.48 | 76.99 | 68.64 | 65.35 | 63.68 | 37.80 | 41.00 | 59.99
MoreauPruner | 17.01 | 30.27 | 66.61 | 77.04 | 68.32 | 65.59 | 65.57 | 38.40 | 41.20 | 60.39
MoreauPruner-GS | 16.65 | 30.69 | 68.87 | 77.26 | 69.81 | 65.04 | 63.64 | 38.23 | 40.60 | 60.49

###  5.1 Experimental Settings

Pre-trained Models To demonstrate the versatility of MoreauPruner across
different scales, we evaluate it on three open-source large language models:
LLaMA-7B[2], LLaMA-13B[2], and Vicuna-7B[14].

Evaluation Building on prior research [7, 6, 30], we assess our method using
seven zero-shot classification tasks on datasets centered around common sense
reasoning: BoolQ [36], PIQA [37], HellaSwag [38], WinoGrande [39], ARC-easy
[40], ARC-challenge [40], and OpenbookQA [41]. Consistent with [42], the model
ranks options in multiple-choice tasks or generates answers for open-ended
questions. Furthermore, we perform a zero-shot perplexity (PPL) analysis on
WikiText2 [43] and PTB [44] with 128-token segments, aligning our methodology
with that of [7, 6].

Implementation Details Aligning with the protocols of the closely related
gradient-based method [6], our model pruning utilizes a calibration set of ten
randomly selected, 128-token truncated sentences from the Bookcorpus [45]. The
gradient of the Moreau envelope is computed using this calibration set, with
the optimization step length fixed at ten. The pruning process typically
completes in approximately 30 minutes on CPUs. In the post-training phase, a
refined version of the Alpaca[46] dataset comprising about 50,000 samples is
employed, with training extending over two epochs and generally taking three
hours on a single NVIDIA RTX 3090 Ti GPU for 7B models. Detailed hyper-
parameter selections are available in the Appendices.

Structural Pruning Baselines We compare MoreauPruner against two fundamental
pruning techniques: Magnitude and Random. Magnitude pruning evaluates weight
significance based on the magnitude of the weight matrix, whereas Random
pruning indiscriminately removes weights. Additionally, we benchmark against
three advanced alternatives: LLM-Pruner [6], which uses a gradient-based
metric to determine weight importance; LoraPrune [7], which utilizes a
LoRA[35]-guided pruning criterion; and WANDA [30], designed for unstructured
or semi-structured pruning but adaptable to other structural frameworks.

We also introduce SmoothGrad, a preliminary version of MoreauPruner that
enhances network smoothness by applying Gaussian smoothing during the
inference, as we explained in Theorem 1. The importance scores are estimated
with the smoothed gradient using Equation 3, and the we still remove those
parameter groups with lower importance scores. A thorough comparison of these
methods is documented in Table 8.

Table 3: Zero-shot performance of the compressed Vicuna-7B.

Pruning Ratio | Method | WikiText2($\downarrow$) | PTB($\downarrow$) | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average
---|---|---|---|---|---|---|---|---|---|---|---
Ratio = 0% | Vicuna-7B† | 16.11 | 61.39 | 76.54 | 77.20 | 70.70 | 67.25 | 65.15 | 41.30 | 40.80 | 62.71
Ratio = 20% w/o tune | Magnitude† | 3539.98 | 5882.21 | 55.90 | 56.15 | 32.37 | 51.85 | 30.01 | 28.41 | 28.20 | 40.41
Random† | 34.63 | 112.44 | 61.47 | 70.89 | 54.67 | 56.27 | 55.60 | 31.74 | 34.60 | 52.18
LLM-Pruner∗[6] | 25.74 | 92.87 | 61.62 | 74.76 | 63.76 | 56.20 | 63.22 | 36.69 | 37.00 | 56.18
SmoothGrad | 25.99 | 92.87 | 60.73 | 74.97 | 63.75 | 54.22 | 64.90 | 37.03 | 37.60 | 56.17
MoreauPruner | 25.54 | 94.34 | 56.82 | 75.79 | 64.73 | 56.35 | 65.95 | 37.88 | 39.80 | 56.76
MoreauPruner-GS | 30.69 | 108.16 | 61.47 | 75.24 | 66.56 | 61.72 | 57.24 | 37.12 | 38.00 | 56.76
Ratio = 20% w tune | LLM-Pruner∗[6] | 19.47 | 72.33 | 64.43 | 76.44 | 65.39 | 60.46 | 63.22 | 35.92 | 38.20 | 57.72
SmoothGrad | 19.51 | 72.05 | 63.46 | 75.68 | 65.38 | 60.93 | 62.79 | 36.43 | 38.80 | 57.64
MoreauPruner | 19.66 | 73.47 | 63.15 | 76.77 | 65.96 | 60.85 | 65.74 | 37.12 | 40.60 | 58.60
MoreauPruner-GS | 19.13 | 73.76 | 65.41 | 76.99 | 68.17 | 65.27 | 66.37 | 38.23 | 39.80 | 60.03

Table 4: Zero-shot performance of the compressed LLaMA-13B.

Pruning Ratio | Method | WikiText2($\downarrow$) | PTB($\downarrow$) | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average
---|---|---|---|---|---|---|---|---|---|---|---
Ratio = 0% | LLaMA-13B† | 11.58 | 44.54 | 68.47 | 78.89 | 76.24 | 70.09 | 74.58 | 44.54 | 42.00 | 64.97
Ratio = 20% w/o tune | LLM-Pruner∗[6] | 16.43 | 59.96 | 63.00 | 77.53 | 73.79 | 64.33 | 69.07 | 40.96 | 40.60 | 61.33
SmoothGrad | 16.55 | 59.96 | 62.94 | 77.04 | 73.78 | 65.98 | 68.35 | 40.53 | 41.40 | 61.43
MoreauPruner | 16.95 | 61.39 | 62.48 | 77.64 | 73.61 | 66.38 | 67.47 | 39.68 | 40.60 | 61.12
MoreauPruner-GS | 17.11 | 61.39 | 72.97 | 77.53 | 74.44 | 64.09 | 66.08 | 40.44 | 41.40 | 62.42
Ratio = 20% w tune | LLM-Pruner∗[6] | 15.04 | 57.00 | 67.28 | 79.00 | 75.13 | 69.06 | 71.68 | 41.89 | 43.60 | 63.95
SmoothGrad | 15.01 | 56.55 | 66.39 | 79.05 | 74.95 | 69.46 | 71.17 | 42.75 | 43.40 | 63.88
MoreauPruner | 15.52 | 57.44 | 64.86 | 79.22 | 75.07 | 70.48 | 71.68 | 43.60 | 42.80 | 63.96
MoreauPruner-GS | 15.28 | 57.67 | 75.17 | 78.24 | 74.77 | 68.19 | 70.12 | 43.09 | 45.00 | 64.94

###  5.2 Robustness Against Weight Perturbation

As we previously discussed, few-shot gradient-based pruning methods are
significantly influenced by the changes of the gradient. Even minor
differences between FP16 and BF16 can lead to markedly different pruning
outcomes. In contrast, MoreauGrad is theoretically robust against norm-bounded
weight perturbation. To validate this assertion, we adhered to the channel-
wise pruning protocol established in prior research [6], removing a fixed
ratio of channels based on their importance score.

Considering that LLMs are usually trained on BF16 but inferred on FP16, our
experiments were conducted on the LLaMA-7B model using both BF16 and FP16
weight bit formats. We standardized the calibration sample selection during
pruning across different settings to ensure a fair comparison. Upon completing
the pruning process, we performed a zero-shot perplexity (PPL) analysis using
128-token segments on the WikiText2 and PTB datasets and compared the
discrepancies between FP16 and BF16. The findings are presented in Table 1b.

The results indicate that, for MoreauPruner, the performance under different
weight format is closer to each other, which indicates a better consistency of
pruning outcomes. The result demonstrates the robustness of MoreauPruner
against weight perturbations caused by different weight formats.

###  5.3 Zero-shot Performance

We have developed two variants of our method, named MoreauPruner and
MoreauPruner-GS, according to whether the sparsity penalty is applied. These
techniques were tested on three pretrained models: LLaMA-7B, Vicuna-7B, and
LLaMA-13B, with their performance detailed in Tables 2, 3 and 4.

The evaluations indicate that MoreauPruner can effectively maintain the model
performance. For example, with a 20% reduction in parameters on LLaMA-7B,
MoreauPruner and MoreauPruner-GS maintains 95.48% and 95.64% of the original
performance with a quick post-training. For Vicuna-7B and LLaMA-13B,
MoreauPruner-GS surpassed MoreauPruner by a notable margin, thanks to the
structural sparsity introduced during optimization. On Vicuna-7B,
MoreauPruner-GS maintains 96.16% of the original performance. For the larger
model LLaMA-13B, We have noticed that the performance gap between the
compressed and original models is closer than that of the 7B models. After a
quick recovery, the zero-shot accuracy of the compressed with 80% parameters
is nearly equivalent to the original model’s performance (64.94% vs. 64.97%).
Such a phenomenon may indicate more redundant weights in the larger models. In
other words, those huge LLMs ($\geq$13B) can be potentially inferred without a
trade-off on performance.

###  5.4 Further Discussion

In this subsection, we extended our experiment to identify how MoreauPruner
works. We also discussed that with more computational resource, how can
MoreauPruner be further improved.

Effect of Function Smoothing In our preliminary evaluations, we introduced
SmoothGrad to assess the impact of function smoothing. This approach often
matches or exceeds the performance of gradient-based competitors. Notably, on
the benchmark model LLaMA-7B, SmoothGrad outperformed all baseline methods
prior to finetuning. These findings suggest that gradient-based pruning
methods could benefit from function smoothing, as it helps mitigate the
excessive sharpness of certain parameters within the differential space.

Table 5: The effect of larger recovery set.

Pruning Ratio | Method | Recovery Set | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average
---|---|---|---|---|---|---|---|---|---|---
Ratio = 0% | LLaMA-7B† | N/A | 73.18 | 78.35 | 72.99 | 67.01 | 67.45 | 41.38 | 42.40 | 63.25
Ratio = 20% | MoreauPruner-GS | 50k[46] | 68.87 | 77.26 | 69.81 | 65.04 | 63.64 | 38.23 | 40.60 | 60.49
MoreauPruner-GS | 2.59M[47] | 76.97 | 76.82 | 68.51 | 66.30 | 70.88 | 41.89 | 40.80 | 63.17

Table 6: The performance of the MoreauPruner-GS on LLaMA-7B with different
calibration set size.

Pruning Ratio | Calibration | WikiText2($\downarrow$) | PTB($\downarrow$) | Average
---|---|---|---|---
Ratio = 20% w/o tune | 10 | 18.72 | 34.91 | 57.30
1000 | 18.50 | 32.16 | 58.12
Ratio = 20% w tune | 10 | 16.65 | 30.69 | 60.49
1000 | 16.95 | 30.21 | 60.63

Larger Recovery Set. In the primary section of the results, the recovery phase
was conducted on Alpaca[46], utilizing a dataset of 50,000 samples. To
demonstrate the potential enhancement achieved by the pruned model, we carried
out an experiment on a significantly larger dataset[47], consisting of 2.59
million samples. The findings, presented in Table 5, reveal that the
performance of the compressed model closely approximates that of the base
model (63.17% v.s. 63.25%), respectively. These results further substantiate
the hypothesis of the presence of redundant weights in LLMs.

Larger Calibration Set. We enlarge the size of calibration set utilized during
pruning phase. We found a larger calibration set can efficiently improving the
pruning quality. Estimating gradient importance on 1000 samples raise the
average zero-shot accuracy from 57.30% to 58.12% and decrease PPL by 0.22 and
2.75 on WikiText2 and PTB. However, the difference on post-finetuning
performance is shrinking, resulting as only 0.14% difference on average
accuracy.

Results on LLaMA3-8B. We extend our result on LLaMA3-8B[3], a stronger
foundational model that is pre-trained with more high-quality data compared
with previous version. The result can be found in Table 7. MoreauPruner-GS
still works well on stronger LLaMA3-8B without hyper-paramerter modification.
We have noticed that the performance of pruned LLaMA3-8B drops more than that
of LLaMA-7B. This may lead by the fact that the pretraining of LLaMA3-8B is
more sufficient according to official report and there is less redundant model
weight. However, the pruned LLaMA3-8B still beats the original LLaMA-7B by a
noticeable margin (63.25% vs. 65.37%).

Effect of Pruning Ratio We explored the influence of varying pruning ratios as
illustrated in Figure 3. It is evident that our methods consistently work well
across different pruning ratios. This stability underscores the robustness and
effectiveness of our pruning strategies.

Impact of Hyper-parameters. The hyper-parameter $\eta$ controls the ratio of
group-sparsity of MoreauPruner-GS during optimization. We conduct an ablation
study on LLaMA-7B with 20% sparsity to evaluate the impact of different hyper-
parameter values $\eta$. The results illustrated in Figure 3 give the average
0-shot accuracy after finetuning. According to the results, we choose
$\eta$=5e-6 for all the experiments in this paper.

Table 7: Zero-shot performance of the compressed LLaMA3-8B.

Pruning Ratio | Method | WikiText2($\downarrow$) | PTB($\downarrow$) | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average
---|---|---|---|---|---|---|---|---|---|---|---
Ratio = 0% | LLaMA-8B† | 14.14 | 27.98 | 81.35 | 80.79 | 79.17 | 72.53 | 80.09 | 53.41 | 45.00 | 70.33
Ratio = 20% w/o tune | LLM-Pruner[6] | 25.74 | 45.69 | 67.55 | 74.97 | 63.33 | 67.80 | 62.29 | 35.49 | 36.60 | 58.29
MoreauPruner-GS | 25.40 | 43.78 | 73.73 | 75.08 | 64.93 | 68.03 | 66.11 | 39.25 | 37.60 | 60.68
Ratio = 20% w tune | LLM-Pruner[6] | 23.71 | 42.01 | 77.52 | 77.69 | 71.75 | 67.96 | 71.63 | 42.24 | 40.00 | 64.11
MoreauPruner-GS | 22.98 | 39.25 | 76.57 | 78.67 | 73.17 | 69.14 | 74.49 | 43.77 | 41.80 | 65.37

Figure 2: The effect of different pruning ratio.

Figure 3: Ablation study on hyper-paramter $\eta$.

##  6 Conclusion

In this paper, we discussed how minor changes in model weights can lead to
unstable pruning results for large language models (LLMs). To address this
instability, we introduced MoreauPruner, a weight-perturbation structural
pruning method. Our theoretical analysis demonstrates that MoreauPruner is
robust to norm-bounded perturbations. Numerical experiments conducted on well-
known LLMs, such as LLaMA-7B, LLaMA-13B, LLaMA-3-8B, and Vicuna-7B, suggest
that MoreauPruner can efficiently compress LLMs while maintaining their
performance. For future work, we propose combining structural pruning
technology with other model compression methods to accelerate model inference
and reduce computational costs.

Limitations. The authors acknowledge that the number of parameters utilized in
the models for this paper only reach 13B due to limited hardware budget. The
performance of MoreauPruner on extremely large-scale models (e.g., 30B, 70B,
etc.) will be further explored once enough hardware resources are available.

## References

  * [1] Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer.  Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale.  In NeurIPS, 2022.
  * [2] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  arXiv preprint arXiv:2302.13971, 2023.
  * [3] AI@Meta.  Llama 3 model card.  2024\.
  * [4] Song Han, Jeff Pool, John Tran, and William Dally.  Learning both weights and connections for efficient neural network.  In NeurIPS, 2015.
  * [5] Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li.  Learning structured sparsity in deep neural networks.  In NeurIPS, 2016.
  * [6] Xinyin Ma, Gongfan Fang, and Xinchao Wang.  Llm-pruner: On the structural pruning of large language models.  In NeurIPS, 2023.
  * [7] Mingyang Zhang, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, Bohan Zhuang, et al.  Pruning meets low-rank parameter-efficient fine-tuning.  arXiv preprint arXiv:2305.18403, 2023.
  * [8] Yann LeCun, John Denker, and Sara Solla.  Optimal brain damage.  Advances in neural information processing systems, 2, 1989.
  * [9] Babak Hassibi and David Stork.  Second order derivatives for network pruning: Optimal brain surgeon.  Advances in neural information processing systems, 5, 1992.
  * [10] Jean-Jacques Moreau.  Proximité et dualité dans un espace hilbertien.  Bulletin de la Société mathématique de France, 93:273–299, 1965.
  * [11] Jingwei Zhang and Farzan Farnia.  Moreaugrad: Sparse and robust interpretation of neural networks via moreau envelope.  ICCV, 2023.
  * [12] Canh T Dinh, Nguyen Tran, and Josh Nguyen.  Personalized federated learning with moreau envelopes.  Advances in Neural Information Processing Systems, 33:21394–21405, 2020.
  * [13] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [14] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.
  * [15] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.  Scaling laws for neural language models.  arXiv preprint arXiv:2001.08361, 2020.
  * [16] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  arXiv preprint arXiv:1503.02531, 2015.
  * [17] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  In NeurIPS, 2019.
  * [18] Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu.  Patient knowledge distillation for bert model compression.  arXiv preprint arXiv:1908.09355, 2019.
  * [19] Siqi Sun, Zhe Gan, Yuwei Fang, Yu Cheng, Shuohang Wang, and Jingjing Liu.  Contrastive distillation on intermediate representations for language model compression.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 498–508, 2020.
  * [20] Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han.  Smoothquant: Accurate and efficient post-training quantization for large language models.  In ICML, pages 38087–38099. PMLR, 2023.
  * [21] Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He.  Zeroquant: Efficient and affordable post-training quantization for large-scale transformers.  Advances in Neural Information Processing Systems, 35:27168–27183, 2022.
  * [22] Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat.  Q8bert: Quantized 8bit bert.  In 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing-NeurIPS Edition (EMC2-NIPS), pages 36–39. IEEE, 2019.
  * [23] Jonathan Frankle and Michael Carbin.  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  In ICLR, 2018.
  * [24] Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang.  Depgraph: Towards any structural pruning.  In CVPR, pages 16091–16101, 2023.
  * [25] Seungcheol Park, Hojun Choi, and U Kang.  Accurate retraining-free pruning for pretrained encoder-based language models.  In ICLR, 2023.
  * [26] Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, and Danqi Chen.  Sheared llama: Accelerating language model pre-training via structured pruning.  arXiv preprint arXiv:2310.06694, 2023.
  * [27] Anna Bair, Hongxu Yin, Maying Shen, Pavlo Molchanov, and Jose Alvarez.  Adaptive sharpness-aware pruning for robust sparse networks.  In ICLR, 2024.
  * [28] Jing Xu and Jingzhao Zhang.  Random masking finds winning tickets for parameter efficient fine-tuning.  In ICML, 2024.
  * [29] Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman.  Slicegpt: Compress large language models by deleting rows and columns.  In ICLR, 2024.
  * [30] Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter.  A simple and effective pruning approach for large language models.  arXiv preprint arXiv:2306.11695, 2023.
  * [31] Guiying Li, Chao Qian, Chunhui Jiang, Xiaofen Lu, and Ke Tang.  Optimization based layer-wise magnitude-based pruning for dnn compression.  In IJCAI, volume 330, pages 2383–2389, 2018.
  * [32] Bryn Elesedy, Varun Kanade, and Yee Whye Teh.  Lottery tickets in linear models: An analysis of iterative magnitude pruning.  arXiv preprint arXiv:2007.08243, 2020.
  * [33] Fang Yu, Kun Huang, Meng Wang, Yuan Cheng, Wei Chu, and Li Cui.  Width & depth pruning for vision transformers.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3143–3151, 2022.
  * [34] Babak Hassibi, David G Stork, and Gregory J Wolff.  Optimal brain surgeon and general network pruning.  In IEEE international conference on neural networks, pages 293–299. IEEE, 1993.
  * [35] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  arXiv preprint arXiv:2106.09685, 2021.
  * [36] Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova.  Boolq: Exploring the surprising difficulty of natural yes/no questions.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2924–2936, 2019.
  * [37] Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al.  Piqa: Reasoning about physical commonsense in natural language.  In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020.
  * [38] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi.  Hellaswag: Can a machine really finish your sentence?  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2019.
  * [39] Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi.  Winogrande: An adversarial winograd schema challenge at scale.  Communications of the ACM, 64(9):99–106, 2021.
  * [40] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord.  Think you have solved question answering? try arc, the ai2 reasoning challenge.  arXiv preprint arXiv:1803.05457, 2018.
  * [41] Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal.  Can a suit of armor conduct electricity? a new dataset for open book question answering.  In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391, 2018.
  * [42] Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou.  A framework for few-shot language model evaluation, 12 2023.
  * [43] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher.  Pointer sentinel mixture models.  In International Conference on Learning Representations, 2016.
  * [44] Mitch Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz.  Building a large annotated corpus of english: The penn treebank.  Computational linguistics, 19(2):313–330, 1993.
  * [45] Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler.  Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.  In Proceedings of the IEEE international conference on computer vision, pages 19–27, 2015.
  * [46] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca, 2023.
  * [47] Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji.  Lamini-lm: A diverse herd of distilled models from large-scale instructions.  arXiv preprint arXiv:2304.14402, 2023.
  * [48] Zinoviy Landsman, Steven Vanduffel, and Jing Yao.  A note on stein’s lemma for multivariate elliptical distributions.  Journal of Statistical Planning and Inference, 143(11):2016–2022, 2013.
  * [49] Xingyu Zhou.  On the fenchel duality between strong convexity and lipschitz continuous gradient.  arXiv preprint arXiv:1803.06573, 2018.

##  Appendix A Proof

###  A.1 Proof of Theorem 1

As Theorem Theorem 1 assumes a Lipschiz function $g$, we can apply the Stein’s
lemma [48] to show

$\displaystyle\nabla g_{\sigma}({\bm{w}})\,=\,\mathbb{E}[\nabla
g({\bm{w}}+{\mathbf{Z}})]\,=\,\mathbb{E}[g({\bm{w}}+{\mathbf{Z}})\frac{{\mathbf{Z}}}{\sigma^{2}}]$

Therefore, for every ${\bm{w}},{\bm{w}}^{\prime}\in\mathbb{R}^{d}$ and
unit-$\ell_{2}$-norm vector $\|{\bm{u}}\|_{2}=1$ we have the following

$\displaystyle\left|{\bm{u}}^{\top}(\nabla g_{\sigma}({\bm{w}})-\nabla
g_{\sigma}({\bm{w}}^{\prime}))\right|=$
$\displaystyle\left|{\bm{u}}^{\top}(\mathbb{E}[\nabla
g({\bm{w}}+{\mathbf{Z}})]-\mathbb{E}[\nabla
g({\bm{w}}^{\prime}+{\mathbf{Z}})])\right|$ $\displaystyle=$
$\displaystyle\left|{\bm{u}}^{\top}(\mathbb{E}\left[\frac{{\mathbf{Z}}}{\sigma^{2}}g({\bm{w}}+{\mathbf{Z}})\right]-\mathbb{E}\left[\frac{{\mathbf{Z}}}{\sigma^{2}}g({\bm{w}}^{\prime}+{\mathbf{Z}})\right])\right|$
$\displaystyle=$
$\displaystyle\left|\mathbb{E}\left[\frac{{\bm{u}}^{\top}{\mathbf{Z}}}{\sigma^{2}}(g({\bm{w}}+{\mathbf{Z}})-g({\bm{w}}^{\prime}+{\mathbf{Z}}))\right]\right|$
$\displaystyle\leq$
$\displaystyle\mathbb{E}\left[\frac{\left|{\bm{u}}^{\top}{\mathbf{Z}}\right|}{\sigma^{2}}\left|g({\bm{w}}+{\mathbf{Z}})-g({\bm{w}}^{\prime}+{\mathbf{Z}})\right|\right]$
$\displaystyle\leq$
$\displaystyle\mathbb{E}\left[\frac{\left|{\bm{u}}^{\top}{\mathbf{Z}}\right|}{\sigma^{2}}\beta\|{\bm{w}}-{\bm{w}}^{\prime}\|_{2}\right]$
$\displaystyle=$
$\displaystyle\frac{\beta\|{\bm{w}}-{\bm{w}}^{\prime}\|_{2}}{\sigma}\mathbb{E}\left[\bigl{|}\frac{{\bm{u}}^{\top}{\mathbf{Z}}}{\sigma}\bigr{|}\right]$
$\displaystyle\leq$
$\displaystyle\frac{\beta\|{\bm{w}}-{\bm{w}}^{\prime}\|_{2}}{\sigma}.$

In the above, note that
$\frac{\mathbf{u}^{\top}\mathbf{Z}}{\sigma}\sim\mathcal{N}(0,1)$. As a result,
the gradient of $g_{\sigma}$ will be $\frac{\beta}{\sigma}$-Lipschitz, and
$g_{\sigma}$ is $\frac{\beta}{\sigma}$-smooth, which means for every
${\bm{w}},{\bm{w}}^{\prime}$ we have,

$\left|g_{\sigma}({\bm{w}}^{\prime})-\nabla
g_{\sigma}({\bm{w}})^{\top}({\bm{w}}^{\prime}-{\bm{w}})\right|\leq\frac{\beta}{2\sigma}\|{\bm{w}}-{\bm{w}}^{\prime}\|_{2}^{2}$

As a result,
$\Theta({\bm{w}})=g_{\sigma}({\bm{w}})+\frac{\beta}{2\sigma}\|{\bm{w}}\|_{2}^{2}$
will be a convex function. Therefore, we can rewrite the definition of the
Moreau envelope as

$\displaystyle g_{\sigma}^{\rho}({\bm{w}})=$
$\displaystyle\min_{\tilde{{\bm{w}}}\in\mathbb{R}^{d}}\Theta(\tilde{{\bm{w}}})-\frac{\beta}{2\sigma}\|\tilde{{\bm{w}}}\|_{2}^{2}+\frac{1}{2\rho}\|\tilde{{\bm{w}}}-{\bm{w}}\|_{2}^{2}$
$\displaystyle=$
$\displaystyle\min_{\tilde{{\bm{w}}}\in\mathbb{R}^{d}}\Theta(\tilde{{\bm{w}}})+(\frac{1}{2\rho}-\frac{\beta}{2\sigma})\|\tilde{{\bm{w}}}\|_{2}^{2}-\frac{1}{\rho}{\bm{w}}^{\top}\tilde{{\bm{w}}}+\frac{1}{2\rho}\|{\bm{w}}\|_{2}^{2}$
$\displaystyle=$
$\displaystyle\frac{1}{2\rho}\|{\bm{w}}\|_{2}^{2}-\frac{1}{\rho}\max_{\tilde{{\bm{w}}}\in\mathbb{R}^{d}}\left\\{{\bm{w}}^{\top}\tilde{{\bm{w}}}-\rho\Theta(\tilde{{\bm{w}}})-\frac{\sigma-\rho\beta}{2\sigma}\|\tilde{{\bm{w}}}\|_{2}^{2}\right\\}.$

Therefore, $\rho g_{\sigma}^{\rho}({\bm{w}})$ is the subtraction of the
Fenchel conjugate of
${\bm{c}}({\bm{w}})=\rho\Theta({\bm{w}})+\frac{\sigma-\rho\beta}{2\sigma}\|\tilde{{\bm{w}}}\|_{2}^{2}$
from the 1-strongly-convex $\frac{1}{2}\|{\bm{w}}\|_{2}^{2}$. Then, we apply
the result that the Fenchel conjugate of a $\mu$-strongly convex function is
$\frac{1}{\mu}$-smooth convex function in [49]. Therefore, the following
Fenchel conjugate

${\bm{c}}^{*}({\bm{w}}):=\max_{\tilde{{\bm{w}}}\in\mathbb{R}^{d}}\left\\{{\bm{w}}^{\top}\tilde{{\bm{w}}}-\rho\Theta(\tilde{{\bm{w}}})-\frac{\sigma-\rho\beta}{2\sigma}\|\tilde{{\bm{w}}}\|_{2}^{2}\right\\}$

is a $\frac{\sigma}{\sigma-\rho\beta}$-smooth convex function. Since, we
subtract two convex functions from each other where the second one has a
constant Hessian $I$, then the resulting function will be smooth of the
following degree:

$\frac{1}{\rho}\mathrm{max}\left\\{|\frac{\sigma}{\sigma-\rho\beta}-1|,|0-1|\right\\}=\frac{\sigma}{\min\\{\sigma\rho,\sigma-\rho\beta\\}},$

which completes the proof of the theorem.

###  A.2 Proof of Theorem 2

To prove Theorem 2, we note that the additional $h$ is a convex function.
Given the formulation of the $h$-Moreau envelope of
$g_{\sigma}^{\rho}({\bm{w}})$ and the assumption $0<\rho<\frac{\sigma}{\beta}$
in the theorem, we have

$\displaystyle g_{\sigma,h}^{\rho}({\bm{w}}):=$
$\displaystyle\min_{\tilde{{\bm{w}}}\in\mathbb{R}^{d}}g_{\sigma}(\tilde{{\bm{w}}})+\frac{1}{2\rho}\|\tilde{{\bm{w}}}-{\bm{w}}\|_{2}^{2}+h(\tilde{{\bm{w}}}-{\bm{w}}),$
$\displaystyle=$
$\displaystyle\min_{\tilde{{\bm{w}}}\in\mathbb{R}^{d}}\Theta(\tilde{{\bm{w}}})+(\frac{1}{2\rho}-\frac{\beta}{2\sigma})\|\tilde{{\bm{w}}}\|_{2}^{2}-\frac{1}{\rho}{\bm{w}}^{\top}\tilde{{\bm{w}}}+\frac{1}{2\rho}\|{\bm{w}}\|_{2}^{2}+h(\tilde{{\bm{w}}}-{\bm{w}}),$

where
$\Theta({\bm{w}})=g_{\sigma}({\bm{w}})+\frac{\beta}{2\sigma}\|{\bm{w}}\|_{2}^{2}$
is a convex function. Then the function
$\phi:\mathbb{R}^{d}\rightarrow\mathbb{R}$ defined as

$\phi(\tilde{{\bm{w}}})=(\frac{1}{2\rho}-\frac{\beta}{2\sigma})\|\tilde{{\bm{w}}}\|_{2}^{2}-\frac{1}{\rho}{\bm{w}}^{\top}\tilde{{\bm{w}}}$

is a $\frac{\sigma-\rho\beta}{\sigma\rho}$-strongly-convex function. As a
result,
$\Theta(\tilde{{\bm{w}}})+\phi(\tilde{{\bm{w}}})+h(\tilde{{\bm{w}}}-{\bm{w}})$
is strongly-convex function with strong-convexity degree
$\frac{\sigma-\rho\beta}{\sigma\rho}$. Therefore, the optimization of
$h$-Moreau envelope has a unique locally and globally optimal solution. we
define the proximal operator of $h$ function as

$\mathrm{prox}_{h(\cdot)}({\bm{w}}):=\operatorname*{arg\,min}_{{\bm{w}}^{\prime}\in\mathbb{R}^{d}}h({\bm{w}}^{\prime})+\frac{1}{2}\|{\bm{w}}^{\prime}-{\bm{w}}\|_{2}^{2}.$

Then since the objective function of $h$-Moreau envelope consists of the
following two convex functions (w.r.t.
$\bm{\delta}:=\tilde{{\bm{w}}}-{\bm{w}}$)
$t_{\bm{w}}(\bm{\delta}):=g_{\sigma}({\bm{w}}+\bm{\delta})+\frac{1}{2\rho}\|\bm{\delta}\|_{2}^{2}$
and $h(\bm{\delta})$, the optimal solution $\bm{\delta}^{*}$ will satisfy the
following equation with $\gamma>0$:

$\bm{\delta}^{*}=\mathrm{prox}_{\gamma
h(\cdot)}\bigl{(}\bm{\delta}^{*}-\gamma\nabla
t_{{\bm{w}}}(\bm{\delta}^{*})\bigr{)}\overset{\gamma=\rho}{=}\mathrm{prox}_{\rho
h(\cdot)}\bigl{(}-\rho\nabla g_{\sigma}({\bm{w}}+\bm{\delta}^{*})\bigr{)}.$

The above implies that, if we use $\psi$ to denote the identity map we will
get:

$\bm{\delta}^{*}({\bm{w}})=\left((\psi+\mathrm{prox}_{\rho
h(\cdot)}\circ\rho\nabla g_{\sigma})^{-1}-\psi\right)({\bm{w}}).$

Note that in the above $\psi+\mathrm{prox}_{\rho h(\cdot)}\circ\rho\nabla
g_{\sigma}$ will be a $(1-\frac{\rho\beta}{\sigma})$-monotone operator, where
we call $t:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ $\tau$-monotone if for
every ${\bm{w}},{\bm{v}}\in\mathbb{R}^{d}$:

$({\bm{v}}-{\bm{w}})^{\top}\bigl{(}t({\bm{v}})-t({\bm{w}})\bigr{)}\geq\tau\|{\bm{v}}-{\bm{w}}\|_{2}^{2}.$

The monotonicity arises because the gradient of a $\lambda$-weakly convex
function is -$\lambda$-monotone, and the proximal operator is known to be
1-monotone. Hence, $\bm{\delta}^{*}({\bm{w}})$ will be a Lipschitz function
with the following Lipschitz constant (note that $(\psi+\mathrm{prox}_{\rho
h(\cdot)}\circ\rho\nabla g_{\sigma})^{-1}$ is a monotone function with a
degree between 0 and $\frac{\sigma}{\sigma-\rho\beta}$):

$\mathrm{max}\left\\{|\frac{\sigma}{\sigma-\rho\beta}-1|,|0-1|\right\\}=\mathrm{max}\left\\{\frac{\rho\beta}{\sigma-\rho\beta},1\right\\}.$

Therefore, for any given convex function $h$, the $h$-MoreauGrad

$h\text{-}\mathrm{MG}^{\rho}[g](\mathbf{w}):=\frac{1}{\rho}\bm{\delta}^{*}({\bm{w}})$

will be a Lipschitz function with the constant
$\frac{\sigma}{\mathrm{min}\left\\{\sigma\rho,\sigma-\rho\beta\right\\}}$.
Then the proof the theorem is finished.

##  Appendix B Experiment Details & Extra Results

###  B.1 A detailed comparison of methods

Table 8: A detailed comparison between methods.

Method | Pruning Criterion | Calibration Set (Size) | Post-Training Set (Size) | Iteratively Pruning | Smoothness
---|---|---|---|---|---
Random | random | N/A | N/A | ✗ | ✗
Magnitude | $\|{\bm{w}}_{i}\|_{2}$ | N/A | N/A | ✗ | ✗
WANDA[30] | $|{\bm{w}}^{k}|\|x_{i}\|_{2}$ | C4(0.128k) | C4(20k) | ✗ | ✗
LoRAPrune[7] | $(\text{LoRA-guided }\frac{\partial L({\bm{w}},\mathcal{D})}{\partial{\bm{w}}^{k}}){\bm{w}}^{k}$ | C4(20k) | C4(20k) | ✓ | ✗
LLM-Pruner[6] | $\frac{\partial L({\bm{w}},\mathcal{D})}{\partial{\bm{w}}^{k}}{\bm{w}}^{k}$ | Bookcorpus(0.01k) | Alpaca(50k) | ✗ | ✗
SmoothGrad | $\mathbb{E}\frac{\partial L({\bm{w}}+{\bm{z}},\mathcal{D})}{\partial{\bm{w}}^{k}}{\bm{w}}^{k}$ | Bookcorpus(0.01k) | Alpaca(50k) | ✗ | ✓
MoreauPruner | $\text{MG}_{\rho}\left(L({\bm{w}},\mathcal{D})\right){\bm{w}}^{k}$ | Bookcorpus(0.01k) | Alpaca(50k) | ✗ | ✓
MoreauPruner-GS | $\text{GS-MG}_{\rho,\eta}\left(L({\bm{w}},\mathcal{D})\right){\bm{w}}^{k}$ | Bookcorpus(0.01k) | Alpaca(50k) | ✗ | ✓

We list the comparison on the experiment setting utilized in our baselines,
which can be found in Table 8.

###  B.2 Parameters Choosing

In the pruning stage, for SmoothGrad, we randomly pick a batch from BookCorpus
[45] with ten 128-token truncated sentences. We then pass the batch to the
model 100 times. We utilized a element-wised Gaussian smoothing, i.e., for
weight parameter $w^{(k)}$, the intensity of Gaussian is
$\sigma=0.05|w^{(k)}|$, where $|\cdot|$ denotes the absolute value function.
The smooth gradient is empirically calculated by averaging the gradients of
each forward pass.

For both MoreauPruner and MoreauPruner-GS, we also apply the element-wise
Gaussian smoothing to the model weights during the optimization of the
gradient of the Moreau Envelope, as SmoothGrad does. The hyper-parameter
$\rho$ is set to 0.05 for MoreauPruner and 0.2 for MoreauPruner-GS. The
stepsize $\gamma$ used in the optimization of the gradient of the Moreau
Envelope is 1e-3 for MoreauPruner and 2e-4 for MoreauPruner-GS. The hyper-
parameter $\eta$ is set to 5e-6 as explained in the main text. We conducted a
parameter search on LLaMA-7B to find suitable hyper-parameters.

In the fine-tuning stage, we use the protocol from previous work [6]. The
batch size is 64. The learning rate is 1e-4 for Alpaca [46] and 5e-5 for
Lamini [47]. The training length is two epochs for Alpaca and three epochs for
Lamini.

###  B.3 Extra Results

Compared with Scratch Training. We compare our MoreauPruner-GS with
StableLM-3B111https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b with
a similar parameter size. With MoreauPruner-GS, We prune LLaMA-7B and get a
compact model with 3.45B parameters. Both models are finetuned on Alpaca[46]
dataset for a fair comparison. The result can be found in Table 9.
MoreauPruner-GS sometimes achieves better results compared with LLMs that are
trained from scratch. We also recognize that the pruned model may not
consistently surpass other models with similar scale, due to the significant
disparity in the size of the training corpus.

Table 9: Comparison between scratch-training and LLaMA-3B obtained by
MoreauPruner-GS

Method | #Param | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | OBQA | Average
---|---|---|---|---|---|---|---|---|---
StableLM-3B† | 3.6B | 48.78 | 69.48 | 44.52 | 54.62 | 50.93 | 25.17 | 27.40 | 45.84
MoreauPruner-GS | 3.5B | 62.26 | 68.39 | 49.58 | 55.72 | 50.97 | 30.20 | 35.40 | 50.36