The Triangle Inequality

If $a$ and $b$ are any two real numbers$,$ then \begin{equation} \label{eq:1.1.3} |a+b|\le |a|+|b|. \end{equation}


If a nonempty set $S$ of real numbers  is bounded above$,$ then $\sup S$ is the unique real number $\beta$ such that \begin{alist} \item % (a)  $x\le\beta$ for all $x$ in $S;$ \item % (b)  if $\epsilon>0$ $($no matter how small$)$$,$ there is an $x_0$ in $S$ such that $x_0> \beta-\epsilon.$ \end{alist}


If $\rho$ and $\epsilon$ are positive$,$ then $n\epsilon>\rho$ for some integer $n.$


The rational numbers are dense in the reals$\,;$ that is, if $a$ and $b$ are real numbers with $a<b,$ there is a rational number $p/q$ such that $a<p/q<b$.


The set of irrational numbers is dense in the reals$\,;$ that is, if $a$ and $b$ are real numbers with $a<b,$  there is an irrational number $t$ such that $a<t<b.$


If a nonempty set $S$ of real numbers  is bounded below$,$ then $\inf S$ is the unique real number $\alpha$ such that \begin{alist} \item % \part{a}  $x\ge\alpha$ for all $x$ in $S;$ \item % (b)  if $\epsilon>0$ $($no matter how small$\,)$, there is an $x_0$ in $S$ such that $x_0< \alpha+\epsilon.$ \end{alist}


Principle of Mathematical Induction

Let $P_1,$ $P_2, $\dots$,$ $P_n,$ \dots\ be propositions$,$ one for each positive integer$,$ such that \begin{alist} \item % (a)  $P_1$ is true$;$ \item % (b)  for each positive integer $n,$  $P_n$  implies $P_{n+1}.$ \end{alist} Then $P_n$ is true for each positive integer $n.$


Let $n_0$ be any integer $($positive$,$  negative$,$ or zero$)$$.$ Let $P_{n_0},$ $P_{n_0+1},$ \dots$,$ $P_n,$ \dots\ be propositions$,$  one for each integer $n\ge n_0,$ such that \begin{alist} \item % (a)  $P_{n_0}$ is true$\,;$ \item % (b)  for each integer $n\ge n_0,$ $P_n$  implies $P_{n+1}.$ \end{alist} Then $P_n$ is true for every integer  $n\ge n_0.$


Let $n_0$ be any integer $($positive$,$  negative$,$ or zero$)$$.$ Let $P_{n_0},$ $P_{n_0+1}, $\dots$,$ $P_n,$ \dots\ be propositions$,$  one for each integer $n\ge n_0,$ such that \begin{alist} \item % (a)  $P_{n_0}$ is true$\,;$ \item % (b) for $n\ge n_0,$ $P_{n+1}$ is true if $P_{n_0},$ $P_{n_0+1}, $\dots$,$ $P_n$ are all true. \end{alist} Then $P_n$ is true for $n\ge n_0.$


\begin{alist} \item % (a)   The union of open sets is open$.$ \item % (b)  The intersection of closed sets is closed$.$ \end{alist} These statements apply to arbitrary collections, finite or infinite, of open and closed sets$.$


no point of $S^c$ is a limit point of~$S.$


If ${\mathcal H}$ is an open covering of a closed and bounded subset $S$ of the real line$,$ then $S$ has an open covering $\widetilde{\mathcal H}$ consisting of finitely many open sets belonging to ${\mathcal H}.$


Every bounded infinite set of real numbers has at least one limit point$.$


then it is unique$\,;$ that is$,$ if \begin{equation} \label{eq:2.1.7} \lim_{x\to x_0} f(x)=L_1\mbox{\quad and \quad}\lim_{x\to x_0} f(x)= L_2, \end{equation} then $L_1=L_2.$


\begin{equation}\label{eq:2.1.9} \lim_{x\to x_0} f(x)=L_1\mbox{\quad and \quad}\lim_{x\to x_0} g(x)= L_2, \end{equation} then \begin{eqnarray} \lim_{x\to x_0} (f+g)(x)\ar= L_1+L_2,\label{eq:2.1.10}\\ \lim_{x\to x_0} (f-g)(x)\ar= L_1-L_2,\label{eq:2.1.11}\\ \lim_{x\to x_0} (fg)(x)\ar= L_1L_2,\label{eq:2.1.12}\\ \arraytext{and, if $L_2\ne0$,}\\ \lim_{x\to x_0}\left(\frac{f}{g}\right)(x)\ar= \frac{L_1}{ L_2}.\label{eq:2.1.13} \end{eqnarray}


A function $f$ has a limit at $x_0$ if and only if it has left- and right-hand limits at $x_0,$ and they are equal.  More specifically$,$ $$ \lim_{x\to x_0} f(x)=L $$ if and only if $$ f(x_0+)=f(x_0-)=L. $$


Suppose that $f$ is monotonic on $(a,b)$ and define $$ \alpha=\inf_{a<x<b}f(x) \mbox{\quad and \quad} \beta=\sup_{a<x<b}f(x). $$ \begin{alist} \item % (a)  If $f$ is nondecreasing$,$ then $f(a+)=\alpha$ and  $f(b-)=\beta.$ \item % (b) If $f$ is nonincreasing$,$ then $f(a+)=\beta$ and $f(b-)=\alpha.$ \\ $($Here $a+=-\infty$ if $a=-\infty$ and $b-=\infty$ if $b=\infty.)$ \item %  (c)  If $a<x_0<b$, then $f(x_0+)$ and $f(x_0-)$ exist and are finite$\,;$ moreover$,$ $$ f(x_0-)\le f(x_0)\le f(x_0+) $$  if $f$ is nondecreasing$,$ and $$ f(x_0-)\ge f(x_0)\ge f(x_0+) $$  if $f$ is nonincreasing$.$ \end{alist}


If $f$ is bounded on $[a,x_0),$ then $\beta=\limsup_{x\to x_0-}f(x)$ exists and is the unique real number with the following properties$\,:$ \begin{alist} \item % (a) If $\epsilon>0$, there is an $a_1$ in $[a,x_0)$ such that \begin{equation} \label{eq:2.1.22} f(x)<\beta+\epsilon\mbox{\quad if \quad} a_1\le x<x_0. \end{equation} \item % (b) If $\epsilon>0$ and $a_1$ is in $[a,x_0),$ then $$ f(\overline x)>\beta-\epsilon\mbox{\quad for some }\overline x\in[a_1,x_0). $$ \end{alist}


If $f$ is bounded on $[a,x_0),$ then $\alpha=\liminf_{x\to x_0-}f(x)$ exists and is the unique real number with the following properties: \begin{alist} \item % (a) If $\epsilon>0,$ there is an $a_1$ in $[a,x_0)$ such that $$ f(x)>\alpha-\epsilon\mbox{\quad if \quad} a_1\le x<x_0. $$ \item % (b) If $\epsilon>0$ and $a_1$ is in $[a,x_0),$ then $$ f(\overline x)<\alpha+\epsilon\mbox{\quad for some }\overline x\in[a_1,x_0). $$ \end{alist}


\vspace*{6pt} \begin{alist} \item % (a) A function $f$ is continuous at $x_0$ if and only if $f$ is defined on an open interval $(a,b)$ containing $x_0$ and for each $\epsilon>0$ there is a $\delta >0$ such that \begin{equation}\label{eq:2.2.1} |f(x)-f(x_0)|<\epsilon \end{equation} whenever $|x-x_0|<\delta.$ \item % (b) A function $f$ is continuous from the right at $x_0$ if and only if $f$ is defined on an interval $[x_0,b)$ and for each $\epsilon> 0$ there is a $\delta>0$ such that $\eqref{eq:2.2.1}$ holds whenever $x_0\le x<x_0+ \delta.$ \item % (c) A function $f$ is continuous from the left at $x_0$ if and only if $f$ is defined on an interval $(a,x_0]$ and for each $\epsilon >0$ there is a $\delta>0$ such that $\eqref{eq:2.2.1}$ holds whenever $x_0-\delta<x\le x_0.$ \end{alist}


If $f$ and $g$ are continuous on a set $S,$ then so are $f+g,$ $f-g,$ and $fg.$ In addition$,$ $f/g$ is continuous at each $x_0$ in $S$ such that $g(x_0)\ne0.$


Suppose that $g$ is continuous at $x_0,$ $g(x_0)$ is an interior point of $D_f,$ and $f$ is continuous at $g(x_0).$ Then  $f\circ g$ is continuous at $x_0.$


If $f$ is continuous on a finite closed interval $[a,b],$ then $f$ is bounded on $[a,b].$


Suppose that $f$ is continuous on a finite closed interval $[a,b].$ Let $$ \alpha=\inf_{a\le x\le b}f(x)\mbox{\quad and \quad}\beta=\sup_{a\le x\le b}f(x). $$ Then $\alpha$ and $\beta$ are respectively the minimum and maximum of $f$ on $[a,b];$ that is$,$  there are points $x_1$ and $x_2$ in $[a,b]$ such that $$ f(x_1)=\alpha\mbox{\quad and \quad} f(x_2)=\beta. $$


Intermediate Value Theorem

Suppose that $f$ is continuous on $[a,b],$ $f(a)\ne f(b),$ and $\mu$ is between $f(a)$ and $f(b).$ Then $f(c)=\mu$ for some $c$ in $(a,b).$


If $f$ is continuous on a closed and bounded interval $[a,b],$ then $f$ is uniformly continuous on $[a,b].$


If $f$ is monotonic and nonconstant on $[a,b],$ then $f$ is continuous on $[a,b]$ if and only if its range $R_f=\set{f(x)}{x\in[a,b]}$ is the closed interval with endpoints $f(a)$ and $f(b).$


Suppose that $f$ is increasing and continuous on $[a,b],$ and let $f(a)=c$ and $f(b)=d.$ Then there is a unique function $g$ defined on $[c,d]$ such that \begin{equation}\label{eq:2.2.17} g(f(x))=x,\quad a\le x\le b, \end{equation} and \begin{equation}\label{eq:2.2.18} f(g(y))=y,\quad c\le y\le d. \end{equation} Moreover$,$ $g$ is continuous and increasing on $[c,d].$


If $f$ is differentiable at $x_0,$ then $f$ is continuous at $x_0.$


If $f$ and $g$ are differentiable at $x_0,$ then so are $f+g,$ $f-g,$ and $fg,$ with \begin{alist} \item % (a)  $(f+g)'(x_0)=f'(x_0)+g'(x_0);$ \item % (b)  $(f-g)'(x_0)=f'(x_0)-g(x_0);$ \item % (c)  $(fg)'(x_0)=f'(x_0)g(x_0)+f(x_0)g'(x_0).$ \end{alist}  The quotient $f/g$ is differentiable at $x_0$ if $g(x_0)\ne0,$ with \begin{alist} \setcounter{lcal}{3} \item % (d) $\dst{\left(\frac{f}{g}\right)' (x_0)= \frac{f'(x_0)g(x_0)-f(x_0)g'(x_0)}{\left[g(x_0)\right]^2}}.$ \end{alist}


The Chain Rule

Suppose that  $g$ is differentiable at $x_0$ and $f$ is differentiable at $g(x_0).$ Then the composite function $h=f\circ g,$ defined by $$ h(x)=f(g(x)), $$ is differentiable at $x_0,$ with $$ h'(x_0)=f'(g(x_0))g'(x_0). $$


If $f$ is differentiable at a local extreme point $x_0\in D_{f}^{0},$ then $f'(x_0)=~0.$


Suppose that  $f$ is continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b),$ and $f(a)=f(b).$ Then $f'(c)=0$ for some $c$ in the open interval $(a,b).$


Intermediate Value Theorem for Derivatives

Suppose that  $f$ is differentiable on $[a,b],$  $f'(a)\ne f'(b),$ and $\mu$ is between $f'(a)$ and $f'(b).$ Then $f'(c)=\mu$ for some $c$ in $(a,b).$


Generalized Mean Value Theorem

If $f$ and $g$ are continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b),$ then \begin{equation} \label{eq:2.3.20} [g(b)-g(a)]f'(c)=[f(b)-f(a)]g'(c) \end{equation} for some $c$ in $(a,b).$


Mean Value Theorem

If $f$ is continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b),$ then $$ f'(c)=\frac{f(b)-f(a)}{ b-a} $$ for some $c$ in $(a,b).$


If $f'(x)=0$ for all $x$ in $(a,b),$ then $f$ is constant on $(a,b).$


If $f'$ exists and does not change sign on $(a,b),$ then $f$ is monotonic on $(a,b):$ increasing$,$ nondecreasing$,$ decreasing$,$ or nonincreasing as $$ f'(x)>0,\quad f'(x)\ge0,\quad f'(x)<0,\mbox{\quad or\quad} f'(x) \le0, $$ respectively$,$ for all $x$ in $(a,b).$


If $$ |f'(x)|\le M,\quad a<x<b, $$ then \begin{equation} \label{eq:2.3.21} |f(x)-f(x')|\le M |x-x'|,\quad x, x'\in (a,b). \end{equation}


Suppose that $f$ and $g$ are differentiable and $g'$ has no zeros on $(a,b).$ Let \begin{equation}\label{eq:2.4.1} \lim_{x\to b-}f(x)=\lim_{x\to b-}g(x)=0 \end{equation} \newpage \noindent or \begin{equation}\label{eq:2.4.2} \lim_{x\to b-}f(x)=\pm\infty\mbox{\quad and \quad} \lim_{x\to b-}g(x)=\pm\infty, \end{equation} and suppose that \begin{equation}\label{eq:2.4.3} \lim_{x\to b-}\frac{f'(x)}{ g'(x)}=L\quad\mbox{$($finite or $\pm \infty)$}. \end{equation} Then \begin{equation}\label{eq:2.4.4} \lim_{x\to b-}\frac{f(x)}{ g(x)}=L. \end{equation}


If $f^{(n)}(x_0)$ exists for some integer $n\ge1$ and $T_n$ is the $n$th Taylor polynomial of $f$ about $x_0,$ then \begin{equation}\label{eq:2.5.5} \lim_{x\to x_0}\frac{f(x)-T_n(x)}{(x-x_0)^n}=0. \end{equation}


Suppose that $f$ has $n$ derivatives at $x_0$ and $n$ is the smallest positive integer such that $f^{(n)}(x_0)\ne 0.$ \begin{alist} \item % (a) If $n$ is odd$,$ $x_0$ is not a local extreme point of $f.$ \item % (b)  If $n$ is even$,$ $x_0$ is a local maximum of $f$ if $f^{(n)}(x_0)<0,$ or a local mininum of $f$ if $f^{(n)}(x_0)>0.$ \end{alist}


Taylor's Theorem

Suppose that $f^{(n+1)}$ exists on an open interval $I$ about $x_0,$ and let $x$ be in $I.$ Then the remainder $$ R_n(x)=f(x)-T_n(x) $$ can be written as $$ R_n(x)=\frac{f^{(n+1)}(c)}{(n+1)!}(x-x_0)^{n+1}, $$ where $c$ depends upon $x$ and is between $x$ and $x_0.$


Extended Mean Value Theorem

Suppose that $f$ is continuous on a finite closed interval $I$ with endpoints $a$ and $b$ $($that is, either $I=(a,b)$ or $I=(b,a)),$ $f^{(n+1)}$ exists on the open interval $I^0,$ and$,$ if $n>0,$ that $f'$, \dots, $f^{(n)}$ exist and are continuous at $a.$ Then \begin{equation}\label{eq:2.5.17} f(b)-\sum_{r=0}^n\frac{f^{(r)}(a)}{ r!}(b-a)^r=\frac{f^{(n+1)}(c)}{(n+1)!} (b-a)^{n+1} \end{equation} for some $c$ in $I^0.$


If $f$ is unbounded on $[a,b],$ then $f$ is not integrable on $[a,b].$


Let $f$ be bounded on $[a,b]$,  and let $P$ be a partition of $[a,b].$  Then \begin{alist} \item % (a)  The upper sum $S(P)$ of $f$ over $P$ is the supremum  of the set of all Riemann sums of $f$ over $P.$ \item % (b)  The lower sum $s(P)$ of $f$ over $P$ is the infimum  of the set of all Riemann sums of $f$ over $P.$ \end{alist}


If $f$ is bounded on $[a,b],$ then \begin{equation} \label{eq:3.2.6} \underline{\int_a^b}f(x)\,dx\le\overline{\int_a^b}f(x)\,dx. \end{equation}


If $f$ is integrable on $[a,b],$ then $$ \underline{\int_a^b}f(x)\,dx=\overline{\int_a^b}f(x)\,dx=\int_a^b f(x)\,dx. $$


If $f$ is bounded on $[a,b]$ and \begin{equation} \label{eq:3.2.16} \underline{\int_a^b} f(x)\,dx=\overline{\int_a^b}f(x)\,dx=L, \end{equation} then $f$ is integrable on $[a,b]$ and \begin{equation} \label{eq:3.2.17} \int_a^b f(x)\,dx=L. \end{equation}


A bounded function $f$ is integrable on $[a,b]$ if and only if $$ \underline{\int_a^b}f(x)\,dx=\overline{\int_a^b}f(x)\,dx. $$


If $f$ is bounded on $[a,b],$ then $f$ is integrable on $[a,b]$  if and only if for each $\epsilon>0$ there is a partition $P$ of $[a,b]$ for which \begin{equation} \label{eq:3.2.19} S(P)-s(P)<\epsilon. \end{equation}


If $f$ is continuous on $[a,b],$ then $f$ is integrable on $[a,b]$.


If $f$ is monotonic on $[a,b],$ then $f$ is integrable on $[a,b]$.


If $f$ and $g$ are integrable on $[a,b],$ then so is $f+g,$ and \vskip4pt $$ \int_a^b (f+g)(x)\,dx=\int_a^b f(x)\,dx+\int_a^b g(x)\,dx. $$


If $f$ is integrable on $[a,b]$ and $c$ is a constant$,$ then $cf$ is integrable on $[a,b]$ and $$ \int_a^b cf(x)\,dx=c\int_a^b f(x)\,dx. $$


If $f_1,$ $f_2,$ \dots$,$ $f_n$ are integrable on $[a,b]$ and $c_1,$ $c_2,$ \dots$,$ $c_n$ are constants$,$ then $c_1f_1+c_2f_2+\cdots+ c_nf_n$ is integrable on $[a,b]$ and \begin{eqnarray*} \int_a^b (c_1f_1+c_2f_2+\cdots+c_nf_n)(x)\,dx\ar=c_1\int_a^b f_1(x)\,dx +c_2\int_a^b f_2(x)\,dx\\ \ar{}+\cdots+c_n\int_a^b f_n(x)\,dx. \end{eqnarray*}


If $f$ and $g$ are integrable on $[a,b]$ and $f(x)\le g(x)$ for $a\le x\le b,$ then \begin{equation}\label{eq:3.3.1} \int_a^b f(x)\,dx\le\int_a^b g(x)\,dx. \end{equation}


If $f$ is integrable on $[a,b],$ then so is $|f|$, and \begin{equation} \label{eq:3.3.3} \left|\int_a^b f(x)\,dx\right|\le\int_a^b |f(x)|\,dx. \end{equation}


If $f$ and $g$ are integrable on $[a,b],$ then so is the product $fg.$


First Mean Value Theorem for Integrals

Suppose that $u$ is continuous and $v$ is integrable and nonnegative on $[a,b].$ Then \begin{equation} \label{eq:3.3.8} \int_a^b u(x)v(x)\,dx=u(c)\int_a^b v(x)\,dx \end{equation} for some $c$ in $[a,b]$.


If $f$ is integrable on $[a,b]$ and $a\le a_1<b_1\le b,$ then $f$ is integrable on $[a_1,b_1].$


If $f$ is integrable on $[a,b]$ and $[b,c],$ then $f$ is integrable on $[a,c],$ and \begin{equation} \label{eq:3.3.12} \int_a^cf(x)\,dx=\int_a^bf(x)\,dx+\int_b^cf(x)\,dx. \end{equation}


If $f$ is integrable on $[a,b]$ and $a\le c\le b,$ then the function $F$ defined by $$  F(x)=\int_c^x f(t)\,dt $$  satisfies a Lipschitz condition on $[a,b],$ and is therefore continuous on $[a,b].$


If $f$ is integrable on $[a,b]$ and $a\le c\le b,$ then $F(x)=\int_c^x f(t)\,dt$ is differentiable at any point $x_0$ in $(a,b)$ where $f$ is continuous$,$ with $F'(x_0)=f(x_0).$ If $f$ is continuous from the right at $a,$ then $F_+'(a)=f(a)$. If $f$ is continuous from the left at $b,$ then $F_-'(b)=f(b).$


Suppose that $F$ is continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b),$ and $f$ is integrable on $[a,b].$ Suppose  also that $$ F'(x)=f(x),\quad a<x<b. $$ Then \begin{equation}\label{eq:3.3.14} \int_a^b f(x)\,dx=F(b)-F(a). \end{equation}


Fundamental Theorem of  Calculus

If $f$ is continuous on $[a,b],$ then $f$ has an antiderivative on $[a,b].$ Moreover$,$ if $F$ is any antiderivative of $f$ on $[a,b],$ then $$ \int_a^b f(x)\,dx=F(b)-F(a). $$


Integration by Parts

If $u'$ and $v'$ are integrable on $[a,b],$ then \begin{equation}\label{eq:3.3.16} \int_a^b u(x)v'(x)\,dx=u(x)v(x)\bigg|^b_a-\int_a^b v(x)u'(x)\,dx. \end{equation}


Second Mean Value Theorem for Integrals

Suppose that $f'$ is nonnegative and integrable and $g$ is continuous on $[a,b].$  Then \begin{equation}\label{eq:3.3.17} \int_a^b f(x)g(x)\,dx=f(a)\int_a^c g(x)\,dx+f(b)\int_c^b g(x)\,dx \end{equation} for some $c$ in $[a,b].$


Suppose that the transformation $x=\phi(t)$ maps the interval $c\le t\le d$ into the interval $a\le x\le b,$ with $\phi(c)=\alpha$ and $\phi(d)=\beta,$ and let $f$ be continuous on $[a,b].$ Let $\phi'$ be integrable on $[c,d].$ Then \begin{equation}\label{eq:3.3.21} \int_\alpha^\beta f(x)\,dx=\int_c^d f(\phi(t))\phi'(t)\,dt. \end{equation}


Suppose that $\phi'$ is integrable and $\phi$ is monotonic on $[c,d],$ and the transformation $x=\phi(t)$ maps $[c,d]$ onto $[a,b].$ Let $f$ be bounded on $[a,b].$ Then $$ g(t)=f(\phi(t))\phi'(t) $$ is integrable on $[c,d]$ if and only if $f$ is integrable over $[a,b],$ and in this case $$ \int_a^b f(x)\,dx=\int_c^d f(\phi(t))|\phi'(t)|\,dt. $$


Suppose that   $f_1,$ $f_2,$ \dots$,$ $f_n$  are locally integrable on $[a,b)$ and that $\int_a^bf_1(x)\,dx,$ $\int_a^bf_2(x)\,dx,$ \dots$,$ $\int_a^bf_n(x)\,dx$ converge$.$ Let $c_1,$ $c_2,$ \dots$,$ $c_n$ be constants$.$ Then  $\int_a^b(c_1f+c_2f_1+\cdots+c_nf_n)(x)\,dx$ converges and \begin{eqnarray*} \int_a^b (c_1f_1+c_2f_2+\cdots+c_nf_n)(x)\,dx\ar=c_1\int_a^b f_1(x)\,dx +c_2\int_a^b f_2(x)\,dx\\ \ar{}+\cdots+c_n\int_a^b f_n(x)\,dx. \end{eqnarray*}


If $f$ is nonnegative and locally integrable on $[a,b),$ then $\int_a^b f(x)\,dx$ converges if the function $$ F(x)=\int_a^x f(t)\,dt $$ is bounded on $[a,b)$, and $\int_a^b f(x)\,dx=\infty$ if it is not. These are the only possibilities, and $$ \int_a^b f(t)\,dt=\sup_{a\le x<b}F(x) $$ in either case$.$


Comparison Test

If $f$ and $g$ are locally integrable on $[a,b)$ and \begin{equation}\label{eq:3.4.2} 0\le f(x)\le g(x),\quad a\le x<b, \end{equation} then \vskip3pt \noindent \part{a}\phantom{xxxxxxxxxxxxxxxxxxxx} $\dst\int_a^b f(x)\,dx<\infty \mbox{\quad if\quad}\dst\int_a^b g(x)\,dx<\infty$ \\ \vskip3pt \noindent and\\ \vskip3pt \noindent \part{b}\phantom{xxxxxxxxxxxxxxxxxxxx} $\dst\int_a^b g(x)\,dx= \infty\mbox{\quad if\quad}\dst\int_a^b f(x)\,dx=\infty$.


Suppose that  $f$ and $g$ are locally integrable on $[a,b),$ $g(x)>0$ and $f(x)\ge0$ on some subinterval $[a_1,b)$ of $[a,b),$ and \begin{equation}\label{eq:3.4.3} \lim_{x\to b-}\frac{f(x)}{ g(x)}=M. \end{equation} \begin{alist} \item % (a) If $0<M<\infty,$ then $\int_a^b f(x)\,dx$ and $\int_a^b g(x)\,dx$ converge or diverge together. \item % (b)  If $M=\infty$ and $\int_a^b g(x)\,dx=\infty,$ then $\int_a^b f(x)\,dx=\infty$. \item % (c)  If $M=0$ and $\int_a^b g(x)\,dx<\infty,$ then $\int_a^b f(x)\,dx<\infty$. \end{alist}


If $f$ is locally integrable on $[a,b)$ and $\int_a^b |f(x)|\,dx<\infty,$ then $\int_a^b f(x)\,dx$ converges$;$ that is$,$ an absolutely convergent integral is convergent$.$


Suppose that  $f$ is continuous  and its antiderivative $F(x)=\int_a^x f(t)\,dt$ is bounded on $[a,b).$ Let $g'$ be absolutely integrable on $[a,b),$ and   suppose that \begin{equation}\label{eq:3.4.9} \lim_{x\to b-} g(x)=0. \end{equation} Then $\int_a^b f(x)g(x)\,dx$ converges$.$


Suppose that  $u$ is continuous on $[a,b)$ and $\int_a^bu(x)\,dx$   diverges$.$ Let $v$ be positive and differentiable on $[a,b),$ and  suppose that  $\lim_{x\to b-}v(x)=\infty$ and $v'/v^2$ is absolutely integrable on $[a,b).$ Then $\int_a^b u(x)v(x)\,dx$  diverges$.$


Suppose that  $g$ is monotonic on $[a,b)$ and $\int_a^b g(x)\,dx=\infty.$ Let $f$ be locally integrable on $[a,b)$ and $$ \int_{x_j}^{x_{j+1}} |f(x)|\,dx\ge\rho,\quad j\ge0, $$ for some positive $\rho,$ where $\{x_j\}$ is an increasing infinite sequence of points in $[a,b)$ such that $\lim_{j\to\infty}x_j=b$ and $x_{j+1}-x_j\le M,$ $j\ge0,$ for some $M.$ Then $$ \int_a^b |f(x)g(x)|\,dx=\infty. $$


Suppose that  $\phi$ is monotonic and $\phi'$ is locally integrable on either of the half-open intervals $I=[c,d)$ or $(c,d],$ and let $x=\phi(t)$ map $I$ onto either of the half-open intervals $J=[a,b)$ or $J=(a,b].$ Let $f$ be locally integrable on $J.$ Then the improper integrals $$ \int_a^b f(x)\,dx\mbox{\quad and\quad}\int_c^d f\left(\phi(t)\right)|\phi'(t)|\,dt $$ \newpage \noindent diverge or converge together$,$ in the latter case to the same value. The same conclusion holds if $\phi$ and $\phi'$ have the stated properties only on the open interval $(a,b),$ the transformation $x=\phi(t)$ maps $(c,d)$ onto $(a,b),$ and $f$ is locally integrable on $(a,b).$


Let $f$ be defined on $[a,b].$ Then $f$ is continuous at $x_0$ in $[a,b]$ if and only if $w_f(x_0)=0.$ $($Continuity at $a$ or $b$ means continuity from the right or left, respectively.$)$


A bounded function $f$ is integrable on a finite interval $[a,b]$ if and only if the set $S$ of discontinuities of $f$ in $[a,b]$ is of Lebesgue measure zero$.$


The limit of a convergent sequence is unique$.$


A convergent sequence is bounded$.$


\begin{alist} \item % (a)  If $\{s_n\}$ is nondecreasing$,$ then $\lim_{n\to\infty}s_n=\sup\{s_n\}.$ \item % (b If $\{s_n\}$ is nonincreasing$,$ then $\lim_{n\to\infty}s_n= \inf\{s_n\}.$ \end{alist}


Let $\lim_{x\to\infty} f(x)=L,$ where $L$ is  in the extended reals$,$ and suppose that $s_n=f(n)$ for large $n.$  Then $$ \lim_{n\to\infty}s_n=L. $$


Let \begin{equation}\label{eq:4.1.4} \lim_{n\to\infty} s_n=s\mbox{\quad and\quad}\lim_{n\to\infty} t_n=t, \end{equation} where $s$ and $t$ are finite$.$  Then \begin{equation}\label{eq:4.1.5} \lim_{n\to\infty} (cs_n)=cs \end{equation} if $c$ is a constant$;$ \begin{eqnarray} \lim_{n\to\infty}(s_n+t_n)\ar=s+t,\label{eq:4.1.6}\\ \lim_{n\to\infty}(s_n-t_n)\ar=s-t, \label{eq:4.1.7}\\ \lim_{n\to\infty}(s_nt_n)\ar=st,\label{eq:4.1.8}\\ \arraytext{and}\nonumber\\ \lim_{n\to\infty}\frac{s_n}{ t_n}\ar=\frac{s}{ t}\label{eq:4.1.9} \end{eqnarray} if $t_n$ is nonzero for all $n$ and $t\ne0$.


\begin{alist} \item % (a) If $\{s_n\}$ is bounded above and does not diverge to $-\infty,$ then there is a unique real number $\overline{s}$ such that$,$ if $\epsilon>0,$ \begin{equation}\label{eq:4.1.16} s_n<\overline{s}+\epsilon\mbox{\quad for large $n$} \end{equation} and \begin{equation}\label{eq:4.1.17} s_n>\overline{s}-\epsilon\mbox{\quad for infinitely many  $n$}. \end{equation} \item % (b) If $\{s_n\}$ is bounded below and does not diverge to $\infty,$ then there is a unique real number $\underline{s}$ such that$,$ if $\epsilon >0,$ \begin{equation}\label{eq:4.1.18} s_n>\underline{s}-\epsilon\mbox{\quad for large $n$} \end{equation} and \begin{equation}\label{eq:4.1.19} s_n<\underline{s}+\epsilon\mbox{\quad for infinitely many $n$}. \end{equation} \end{alist}


Every sequence $\{s_n\}$ of real numbers has a unique limit superior$,$ $\overline{s},$ and a unique limit inferior$,$ $\underline{s}$, in the extended reals$,$ and \begin{equation}\label{eq:4.1.21} \underline{s}\le \overline{s}. \end{equation}


If $\{s_n\}$ is a sequence of real numbers, then \begin{equation}\label{eq:4.1.22} \lim_{n\to\infty} s_n=s \end{equation} if and only if \begin{equation}\label{eq:4.1.23} \limsup_{n\to\infty}s_n=\liminf_{n\to\infty} s_n=s. \end{equation}


A sequence $\{s_n\}$ of real numbers converges if and only if$,$ for every  $\epsilon>0,$ there is an integer $N$ such that \begin{equation}\label{eq:4.1.24} |s_n-s_m|<\epsilon\mbox{\quad if\quad} m,n\ge N. \end{equation}


If \begin{equation}\label{eq:4.2.1} \lim_{n\to\infty}s_n=s\quad (-\infty\le s\le\infty), \end{equation} then \begin{equation}\label{eq:4.2.2} \lim_{k\to\infty} s_{n_k}=s \end{equation} for every subsequence $\{s_{n_k}\}$ of $\{s_n\}.$


If $\{s_n\}$ is monotonic and has a subsequence $\{s_{n_k}\}$ such that $$ \lim_{k\to\infty} s_{n_k}=s\quad (-\infty\le s\le\infty), $$ then $$ \lim_{n\to\infty} s_n=s. $$


A point $\overline{x}$ is a limit point of a set $S$ if and only if there is a sequence $\{x_n\}$ of points in $S$ such that $x_n\ne\overline{x}$ for $n\ge 1,$ and $$ \lim_{n\to\infty}x_n=\overline{x}. $$


\vspace*{3pt} \begin{alist} \item % (a)   If $\{x_n\}$ is bounded$,$ then $\{x_n\}$ has a convergent subsequence$.$ \vspace*{3pt} \item % (b)   If $\{x_n\}$ is unbounded above$,$  then $\{x_n\}$ has a subsequence $\{x_{n_k}\}$ such that $$ \lim_{k\to\infty} x_{n_k}=\infty. $$ \vspace*{3pt} \item % (c)   If $\{x_n\}$ is unbounded below$,$ then $\{x_n\}$ has a subsequence $\{x_{n_k}\}$ such that $$ \lim_{k\to\infty} x_{n_k}=-\infty. $$ \end{alist}


Let $f$ be defined on a closed interval $[a,b]$ containing $\overline{x}.$ Then $f$ is continuous at $\overline{x}$ $($from the right if $\overline{x}=a,$ from the left if $\overline{x}=b$$)$ if and only if \begin{equation}\label{eq:4.2.6} \lim_{n\to\infty} f(x_n)=f(\overline{x}) \end{equation} whenever $\{x_n\}$ is a sequence of points in $[a,b]$ such that \begin{equation}\label{eq:4.2.7} \lim_{n\to\infty} x_n=\overline{x}. \end{equation}


If $f$ is continuous on a closed interval $[a,b],$ then $f$ is bounded on $[a,b].$


The sum of a convergent series is unique$.$


Let $$ \sum_{n=k}^\infty a_n=A\mbox{\quad and\quad}\sum_{n=k}^\infty b_n=B, $$ where $A$ and $B$ are finite$.$  Then $$ \sum_{n=k}^\infty (ca_n)=cA $$ if $c$ is a constant$,$ $$ \sum_{n=k}^\infty (a_n+b_n)=A+B, $$ and $$ \sum_{n=k}^\infty (a_n-b_n)=A-B. $$ These relations also hold if one or both of $A$ and $B$ is infinite, provided that the right sides are not indeterminate$.$


Cauchy's Convergence Criterion for Series

A series $\sum a_n$ converges if and only if for every $\epsilon>0$ there is an integer $N$ such that \begin{equation}\label{eq:4.3.3} |a_n+a_{n+1}+\cdots+a_m|<\epsilon\mbox{\quad if\quad} m\ge n\ge N. \end{equation}


If $a_n\ge0$ for $n\ge k,$ then $\sum a_n$ converges if its partial sums are bounded$,$ or diverges to $\infty$ if they are not$.$ These are the only possibilities and$,$ in either case$,$ $$ \sum_{n=k}^\infty a_n =\,\sup\set{A_n}{n\ge k}\negthickspace, $$ where $$ A_n=a_k+a_{k+1}+\cdots+a_n,\quad n\ge k. $$


The Comparison Test

Suppose that \begin{equation}\label{eq:4.3.5} 0\le a_n\le b_n,\quad n\ge k. \end{equation} Then \begin{alist} \item % (a)  $\sum a_n<\infty$ if  $\sum b_n<\infty$$.$ \item % (b)  $\sum b_n=\infty$  if  $\sum a_n=\infty.$ \end{alist}


The Integral Test

Let \begin{equation}\label{eq:4.3.7} c_n=f(n),\quad n\ge k, \end{equation} where $f$ is  positive$,$ nonincreasing$,$ and locally integrable on $[k,\infty).$ Then \begin{equation}\label{eq:4.3.8} \sum c_n<\infty \end{equation} if and only if \begin{equation}\label{eq:4.3.9} \int^\infty_k f(x)\,dx<\infty. \end{equation}


Suppose that  $a_n\ge0$ and $b_n>0$ for $n\ge k.$ Then \begin{alist} \item % (a) $\dst{\sum a_n<\infty\mbox{\quad if\quad}\sum b_n< \infty\mbox{\quad and\quad}\limsup_{n\to\infty} a_n/b_n<\infty}.$ \item % (b)  $\dst{\sum a_n=\infty\mbox{\quad if\quad}\sum b_n= \infty\mbox{\quad and\quad}\liminf_{n\to\infty} a_n/b_n>0}.$ \end{alist}


Suppose that  $a_n>0,$ $b_n>0,$ and \begin{equation}\label{eq:4.3.12} \frac{a_{n+1}}{ a_n}\le \frac{b_{n+1}}{ b_n}. \end{equation} Then \begin{alist} \item % (a)  $\sum a_n<\infty$ if  $\sum b_n<\infty.$ \item % (b)  $\sum b_n=\infty$  if  $\sum a_n=\infty.$ \end{alist}


The Ratio Test

Suppose that  $a_n>0$ for $n\ge k.$  Then \vspace*{5pt} \begin{alist} \vspace*{5pt} \item % (a) $\sum a_n<\infty$ if\, $\limsup_{n\to\infty} a_{n+1}/a_n<1.$ \vspace*{5pt} \item % (b)  $\sum a_n=\infty$  if\, $\liminf_{n\to\infty} a_{n+1}/a_n>1.$ \end{alist} \vspace*{5pt} \noindent If \begin{equation}\label{eq:4.3.13} \liminf_{n\to\infty}\frac{a_{n+1}}{ a_n}\le1\le \limsup_{n\to\infty}\frac{a_{n+1}}{ a_n}, \end{equation} then the test is inconclusive$;$ that is$,$ $\sum a_n$ may converge or diverge$.$


Suppose that  $a_n>0$ for large $n.$  Let $$ M=\limsup_{n\to\infty} n\left(\frac{a_{n+1}}{ a_n}- 1\right)\mbox{\quad and\quad} m=\liminf_{n\to\infty} n \left(\frac{a_{n+1}}{ a_n}-1\right). $$ Then \begin{alist} \item % (a)  $\sum a_n<\infty$ if $M<-1.$ \item % (b)  $\sum a_n=\infty$  if $m>-1.$ \end{alist} The test is inconclusive if $m\le-1\le M.$


Cauchy's Root Test

If $a_n\ge 0$ for $n\ge k,$ then \begin{alist} \item % (a)  $\sum a_n<\infty$ if $\limsup_{n\to\infty} a^{1/n}_n<1.$ \item % (b)  $\sum a_n=\infty$ if $\limsup_{n\to\infty} a^{1/n}_n>1.$ \end{alist} The test is inconclusive if $\limsup_{n\to\infty} a^{1/n}_n= 1.$


absolutely$,$ then $\sum a_n$ converges$.$


Dirichlet's Test for Series

The series $\sum ^\infty_{n=k} a_nb_n$ converges if $\lim_{n\to\infty} a_n= 0,$ \begin{equation}\label{eq:4.3.18} \sum |a_{n+1}-a_n|<\infty, \end{equation} and \begin{equation}\label{eq:4.3.19} |b_k+b_{k+1}+\cdots+b_n|\le M,\quad n\ge k, \end{equation} for some constant $M.$


Suppose that  $\sum_{n=k}^\infty a_n=A,$ where $-\infty \le A\le\infty.$ Let $\{n_j\}_1^\infty$ be an increasing sequence of integers, with $n_1\ge k$. Define \begin{eqnarray*} b_1\ar=a_k+\cdots+a_{n_1},\\ b_2\ar=a_{{n_1}+1}+\cdots+a_{n_2},\\ &\vdots\\ b_r\ar=a_{n_{r-1}+1}+\cdots+a_{n_r}. \end{eqnarray*} Then $$ \sum_{j=1}^\infty b_{n_j}=A. $$


If $\sum_{n=1}^\infty b_n$ is a rearrangement of an absolutely convergent series $\sum_{n=1}^\infty a_n,$ then $\sum_{n=1}^\infty b_n$ also converges absolutely$,$ and to the same sum$.$


If $P=\{a_{n_i}\}_1^\infty$ and $Q= \{a_{m_j}\}_1^\infty$ are respectively the subsequences of all positive and negative terms in a conditionally convergent series $\sum a_n,$ then \begin{equation} \label{eq:4.3.24} \sum_{i=1}^\infty a_{n_i}=\infty\mbox{\quad and\quad}\sum_{j=1}^\infty a_{m_j}=-\infty. \end{equation}


Suppose that $\sum_{n=1}^\infty a_n$ is conditionally convergent and  $\mu$ and $\nu$ are arbitrarily given in  the extended reals$,$ with $\mu\le\nu.$ Then the terms of $\sum_{n=1}^\infty a_n$ can be rearranged  to form a series $\sum_{n=1}^\infty b_n$ with partial sums $$ B_n=b_1+b_2+\cdots+b_n,\quad n\ge1, $$ such that \begin{equation}\label{eq:4.3.25} \limsup_{n\to\infty}B_n=\nu\mbox{\quad and\quad} \liminf_{n\to\infty}B_n=\mu. \end{equation}


Let $$ \sum_{n=0}^\infty a_n=A\mbox{\quad and\quad}\sum_{n=0}^\infty b_n=B, $$ where $A$ and $B$ are finite, and at least one term of each series is nonzero. Then $\sum_{n=0}^\infty p_n=AB$ for every sequence $\{p_n\}$ obtained by ordering the products in $\eqref{eq:4.3.33}$ if and only if $\sum a_n$ and $\sum b_n$ converge absolutely$.$ Moreover$,$ in this case, $\sum p_n$ converges absolutely$.$


If $\sum_{n=0}^\infty a_n$ and $\sum_{n=0}^\infty b_n$ converge absolutely to sums $A$ and $B,$ then the Cauchy product of $\sum_{n=0}^\infty a_n$ and $\sum_{n=0}^\infty b_n$ converges absolutely to $AB.$


Let $\{F_n\}$ be defined on $S.$ Then \begin{alist} \item % (a) $\{F_n\}$ converges pointwise to $F$ on $S$ if and only if there is, for each $\epsilon>0$ and $x\in S$, an integer $N$ $($which may depend on $x$ as well as $\epsilon)$ such that $$ |F_n(x)-F(x)|<\epsilon\mbox{\quad if\quad}\ n\ge N. $$ \item % (b)  $\{F_n\}$ converges uniformly to $F$ on $S$ if and only if there is for each $\epsilon>0$ an integer $N$ $($which depends only on $\epsilon$ and not on any particular $x$ in $S)$ such that $$ |F_n(x)-F(x)|<\epsilon\mbox{\quad for all $x$ in $S$ if $n\ge N$}. $$ \end{alist}


If $\{F_n\}$ converges uniformly to $F$ on $S,$ then $\{F_n\}$ converges pointwise to $F$ on $S.$ The converse is false$;$ that is$,$ pointwise convergence does not imply uniform convergence.


Cauchy's Uniform Convergence Criterion

A sequence of functions $\{F_n\}$ converges uniformly on a set $S$ if and only if for each $\epsilon>0$ there is an integer $N$ such that \begin{equation} \label{eq:4.4.2} \|F_n-F_m\|_S<\epsilon\mbox{\quad if\quad} n, m\ge N. \end{equation}


If $\{F_n\}$ converges uniformly to $F$ on $S$ and each $F_n$ is continuous at a point $x_0$ in $S,$ then so is $F$. Similar statements hold for continuity from the right and left$.$


Suppose that  $\{F_n\}$ converges uniformly to $F$ on $S=[a,b]$. Assume that $F$ and all $F_n$ are   integrable on $[a,b].$ Then \begin{equation} \label{eq:4.4.10} \int_a^b F(x)\,dx=\lim_{n\to\infty}\int_a^b F_n(x)\,dx. \end{equation}


Suppose that  $\{F_n\}$ converges pointwise to $F$ and each $F_n$ is integrable on $[a,b].$ \begin{alist} \item % (a) If the convergence is uniform$,$ then $F$ is integrable on $[a,b]$ and $\eqref{eq:4.4.10}$ holds. \item % (b) If the sequence $\{\|F_n\|_{[a,b]}\}$ is bounded and $F$ is integrable on $[a,b],$ then $\eqref{eq:4.4.10}$ holds. \end{alist}


Suppose that  $F'_n$ is continuous on $[a,b]$ for all $n\ge1$ and $\{F'_n\}$ converges uniformly on $[a,b].$ Suppose also that  $\{F_n(x_0)\}$ converges for some $x_0$ in $[a,b].$ Then $\{F_n\}$ converges uniformly on $[a,b]$ to a differentiable limit function $F,$ and \begin{equation} \label{eq:4.4.11} F'(x)=\lim_{n\to\infty}F'_n(x),\quad a<x<b, \end{equation} while \begin{equation} \label{eq:4.4.12} F'_+(a)=\lim_{n\to\infty}F'_n(a+)\mbox{\quad and\quad} F'_-(b)= \lim_{n\to\infty}F'_n(b-). \end{equation}


Cauchy's  Uniform Convergence Criterion

A series $\sum f_n$ converges uniformly on a set $S$ if and only if for each $\epsilon>0$ there is an integer $N$ such that \vskip0pt \begin{equation} \label{eq:4.4.16} \|f_n+f_{n+1}+\cdots+f_m\|_S<\epsilon\mbox{\quad if\quad} m\ge n\ge N. \end{equation}


Weierstrass's Test

The series $\sum f_n$ converges uniformly on $S$ if \begin{equation} \label{eq:4.4.17} \|f_n\|_S\le M_n,\quad n\ge k, \end{equation} where $\sum M_n<\infty.$


Dirichlet's Test for Uniform Convergence

The series $$ \sum_{n=k}^\infty  f_ng_n $$  converges uniformly on $S$ if  $\{f_n\}$ converges uniformly to zero on $S,$  $\sum (f_{n+1}-f_n)$ converges absolutely uniformly on $S,$ and \begin{equation} \label{eq:4.4.19} \|g_k+g_{k+1}+\cdots+g_n\|_S\le M,\quad n\ge k, \end{equation} for some constant $M.$


If $\sum_{n=k}^\infty  f_n$ converges uniformly to $F$ on $S$ and each $f_n$ is continuous at a point $x_0$ in $S,$ then so is $F.$ Similar statements hold for continuity from the right and left$.$


Suppose that  $\sum_{n=k}^\infty f_n$ converges uniformly to $F$ on $S=[a,b].$ Assume that $F$ and  $f_n,$ $n\ge k,$ are   integrable on $[a,b].$ Then $$ \int_a^b F(x)\,dx=\sum_{n=k}^\infty \int_a^b f_n(x)\,dx. $$


Suppose that  $f_n$ is continuously differentiable on $[a,b]$ for each $n\ge k,$ $\sum_{n=k}^\infty f_n(x_0)$ converges for some $x_0$ in $[a,b],$ and $\sum_{n=k}^\infty  f'_n$ converges uniformly on $[a,b].$ Then $\sum_{n=k}^\infty f_n$ converges uniformly on $[a,b]$ to a differentiable function $F,$ and $$ F'(x)=\sum_{n=k}^\infty  f'_n(x),\quad a<x<b, $$ while $$ F'(a+)=\sum_{n=k}^\infty  f'_n(a+)\mbox{\quad and\quad} F'(b-)= \sum_{n=k}^\infty  f'_n(b-). $$


In connection with the power series $\eqref{eq:4.5.1},$ define $R$ in the extended reals by \begin{equation}\label{eq:4.5.2} \frac{1}{ R}=\limsup_{n\to\infty} |a_n|^{1/n}. \end{equation} In particular$,$ $R=0$ if $\limsup_{n\to\infty} |a_n|^{1/n}= \infty$, and $R=\infty$ if $\limsup_{n\to\infty} |a_n|^{1/n}=0.$ Then the power series converges \begin{alist} \item % (a)  only for $x=x_0$ if $R=0;$ \item % (b)  for all $x$ if $R=\infty,$ and absolutely uniformly in every bounded set$;$ \item % (c) for $x$ in $(x_0-R, x_0+R)$ if $0<R<\infty,$ and absolutely uniformly in every closed subset of this interval. \end{alist} The series diverges if $|x-x_0|>R.$ No general statement can be made concerning convergence at the endpoints $x=x_0+R$ and $x=x_0-R:$ the series may converge absolutely or conditionally at both$,$ converge conditionally at one and diverge at the other$,$ or diverge at both$.$


The radius of convergence of $\sum a_n(x-x_0)^n$ is given by $$ \frac{1}{ R}=\lim_{n\to\infty}\left|\frac{a_{n+1}}{a_n}\right| $$ if the limit exists in the extended reals$.$


A power series $$ f(x)=\sum^\infty_{n=0} a_n(x-x_0)^n $$ \newpage \noindent with positive radius of convergence $R$ is continuous and differentiable in its interval of convergence$,$ and its derivative can be obtained by differentiating term by term$;$ that is$,$ \begin{equation} \label{eq:4.5.9} f'(x)=\sum^\infty_{n=1} na_n(x-x_0)^{n-1}, \end{equation} which can also be written as \begin{equation} \label{eq:4.5.10} f'(x)=\sum^\infty_{n=0}(n+1)a_{n+1} (x-x_0)^n. \end{equation} This series also has radius of convergence $R.$


A power series $$ f(x)=\sum_{n=0}^\infty a_n(x-x_0)^n $$ with positive radius of convergence $R$  has derivatives of all orders in its interval of convergence$,$ which can be obtained by repeated term by term differentiation$;$ thus$,$ \begin{equation}\label{eq:4.5.11} f^{(k)}(x)=\sum^\infty_{n=k} n(n-1)\cdots (n-k+1)a_n(x-x_0)^{n-k}. \end{equation} The radius of convergence of each of these series is $R.$


If $x_1$ and $x_2$ are in the interval of convergence of $$ f(x)=\sum^\infty_{n=0} a_n(x-x_0)^n, $$ then $$ \int_{x_1}^{x_2} f(x)\,dx=\sum^\infty_{n=0}\frac{a_n}{ n+1}\left[(x_2-x_0)^{n+1}- (x_1-x_0)^{n+1}\right]; $$ that is$,$ a power series may be integrated term by term between any two points in its interval of convergence$.$


Suppose that  $f$ is infinitely differentiable on an interval $I$ and \begin{equation}\label{eq:4.5.18} \lim_{n\to\infty}\frac{r^n}{ n!}\|f^{(n)}\|_I=0. \end{equation} Then$,$ if $x_0\in I^0,$ the Taylor series $$ \sum^\infty_{n=0}\frac{f^{(n)}(x_0)}{ n!} (x-x_0)^n $$  converges uniformly to $f$ on $$ I_r=I\cap [x_0-r,x_0+r]. $$


If \begin{equation}\label{eq:4.5.22} f(x)=\sum^\infty_{n=0} a_n(x-x_0)^n,\quad |x-x_0|<R_1, \end{equation} \begin{equation}\label{eq:4.5.23} g(x)=\sum^\infty_{n=0} b_n(x-x_0)^n,\quad |x-x_0|<R_2, \end{equation} and $\alpha$ and $\beta$ are constants$,$ then $$ \alpha f(x)+\beta g(x)=\sum^\infty_{n=0} (\alpha a_n+\beta b_n)(x-x_0)^n,\quad |x-x_0|<R, $$ where  $R\ge\min\{R_1,R_2\}.$


If $f$ and $g$ are given by $\eqref{eq:4.5.22}$ and $\eqref{eq:4.5.23},$ then \begin{eqnarray} f(x)g(x)\ar=\sum^\infty_{n=0} c_n(x-x_0)^n,\quad|x-x_0|<R,\label{eq:4.5.24}\\ \arraytext{where}\nonumber c_n\ar=\sum^n_{r=0} a_rb_{n-r}=\sum^n_{r=0} a_{n-r}b_r\nonumber \end{eqnarray} and $R\ge\min\{R_1,R_2\}.$


Abel's Theorem

Let $f$ be defined by a power series $\eqref{eq:4.5.29}$ with finite radius of convergence $R.$ \begin{alist} \item % (a)  If $\sum^\infty_{n=0} a_nR^n$ converges$,$ then $$ \lim_{x\to (x_0+R)-}f(x)=\sum^\infty_{n=0} a_nR^n. $$ \item % (b)  If $\sum^\infty_{n=0} (-1)^na_nR^n$ converges$,$ then $$ \lim_{x\to (x_0-R)+} f(x)=\sum^\infty_{n=0} (-1)^na_nR^n. $$ \end{alist}


If $\mathbf{X},$ $\mathbf{Y},$ and $\mathbf{Z}$ are in $\R^n$ and $a$ and $b$ are real numbers$,$ then \begin{alist} \item % (a)  $\mathbf{X}+\mathbf{Y}=\mathbf{Y}+\mathbf{X}$  $($vector addition is commutative$).$ \item % (b) $(\mathbf{X}+\mathbf{Y})+\mathbf{Z}=\mathbf{X}+(\mathbf{Y}+\mathbf{Z})$ $($vector addition is associative$).$ \item % (c)  There is a unique vector $\mathbf{0},$ called the zero vector$,$ such that $\mathbf{X}+\mathbf{0}=\mathbf{X}$ for all $\mathbf{X}$ in $\R^n.$ \item % (d)  For each $\mathbf{X}$ in $\R^n$ there is a unique vector $-\mathbf{X}$ such that $\mathbf{X}+(-\mathbf{X})=\mathbf{0}.$ \item % (e)  $a(b\mathbf{X})=(ab)\mathbf{X}.$ \item % (f) $(a+b)\mathbf{X}=a\mathbf{X}+b\mathbf{X}.$ \item % (g)  $a(\mathbf{X}+\mathbf{Y})=a\mathbf{X}+a\mathbf{Y}.$ \item % (h)  $1\mathbf{X}=\mathbf{X}.$ \end{alist}


Triangle Inequality

If $\mathbf{X}$ and $\mathbf{Y}$ are in $\R^n,$ then \begin{equation}\label{eq:5.1.6} |\mathbf{X}+\mathbf{Y}|\le |\mathbf{X}|+|\mathbf{Y}|, \end{equation} with equality if and only if one of the vectors is a nonnegative multiple of the other$.$


If $\mathbf{X},$ $\mathbf{Y},$ and $\mathbf{Z}$ are members of $\R^n$ and $a$ is a scalar, then \begin{alist} \item % (a)  $|a\mathbf{X}|=|a|\,|\mathbf{X}|.$ \item % (b)  $|\mathbf{X}|\ge0,$ with equality if and only if $\mathbf{X}= \mathbf{0}.$ \item % (c)  $|\mathbf{X}-\mathbf{Y}|\ge0,$ with equality if and only if $\mathbf{X}=\mathbf{Y}.$ \item % (d) $\mathbf{X}\cdot\mathbf{Y}=\mathbf{Y}\cdot\mathbf{X}.$ \item % (e)  $\mathbf{X}\cdot (\mathbf{Y}+\mathbf{Z})=\mathbf{X}\cdot\mathbf{Y}+ \mathbf{X}\cdot\mathbf{Z}.$ \item % (f)  $(c\mathbf{X})\cdot\mathbf{Y}=\mathbf{X}\cdot (c\mathbf{Y})= c(\mathbf{X}\cdot\mathbf{Y}).$ \end{alist}


Let $$ \overline{\mathbf{X}}=(\overline{x}_1,\overline{x}_2, \dots,\overline{x}_n) \mbox{\quad and\quad}\mathbf{X}_r=(x_{1r}, x_{2r}, \dots, x_{nr}),\quad r\ge1. $$ Then $\lim_{r\to\infty}\mathbf{X}_r=\overline{\mathbf{X}}$ if and only if $$ \lim_{r\to\infty}x_{ir}=\overline{x}_i,\quad 1\le i\le n; $$ that is$,$ a sequence $\{\mathbf{X}_r\}$ of points in $\R^n$ converges to a limit $\overline{\mathbf{X}}$ if and only if the sequences of components of $\{\mathbf{X}_r\}$ converge to the respective components of $\overline{\mathbf{X}}.$


Cauchy's Convergence Criterion

A sequence $\{\mathbf{X}_r\}$  in $\R^n$ converges if and only if for each $\epsilon>0$ there is an integer $K$ such that $$ |\mathbf{X}_r-\mathbf{X}_s|<\epsilon\mbox{\quad if\quad} r,s\ge K. $$


Principle of Nested Sets

If $S_1,$ $S_2,$ \dots\ are closed nonempty subsets of $\R^n$ such that \begin{equation}\label{eq:5.1.14} S_1\supset S_2\supset\cdots\supset S_r\supset\cdots \end{equation} and \begin{equation}\label{eq:5.1.15} \lim_{r\to\infty} d(S_r)=0, \end{equation} then the intersection $$ I=\bigcap^\infty_{r=1}S_r $$ contains exactly one point$.$


Heine--Borel Theorem

If ${\mathcal H}$ is an open covering of a compact subset $S,$  then $S$ can be covered by finitely many sets from ${\mathcal H}.$


An open set $S$ in $\R^n$ is connected if and only if it is polygonally connected$.$


If $\lim_{\mathbf{X}\to\mathbf{X}_0} f(\mathbf{X})$ exists$,$ then it is unique.


Suppose that  $f$ and $g$ are defined on a set $D,$ $\mathbf{X}_0$ is a limit point of $D,$ and $$ \lim_{\mathbf{X}\to\mathbf{X}_0} f(\mathbf{X})=L_1,\quad\lim_{\mathbf{X}\to\mathbf{X}_0} g(\mathbf{X})=L_2. $$ Then \begin{eqnarray} \lim_{\mathbf{X}\to\mathbf{X}_0}(f+g)(\mathbf{X})\ar=L_1+L_2,\label{eq:5.2.10}\\ \lim_{\mathbf{X}\to\mathbf{X}_0}(f-g)(\mathbf{X})\ar=L_1-L_2,\label{eq:5.2.11}\\ \lim_{\mathbf{X}\to\mathbf{X}_0}(fg)(\mathbf{X})\ar=L_1L_2,\label{eq:5.2.12}\\ \arraytext{and$,$ if $L_2\ne0,$}\nonumber\\ \lim_{\mathbf{X}\to\mathbf{X}_0}\left(\frac{f}{ g}\right)(\mathbf{X}) \ar=\frac{L_1}{ L_2}.\label{eq:5.2.13} \end{eqnarray}


Suppose that  $\mathbf{X}_0$ is in $D_f$ and is a limit point of $D_f.$ Then $f$ is continuous at $\mathbf{X}_0$ if and only if for each $\epsilon>0$ there is a $\delta>0$ such that $$ \left|f(\mathbf{X})-f(\mathbf{X}_0)\right|<\epsilon $$ whenever $$ |\mathbf{X}-\mathbf{X}_0|<\delta\mbox{\quad and\quad}\mathbf{X}\in D_f. $$


If $f$ and $g$ are continuous on a set $S$ in $\R^n,$ then so are $f+g,$ $f-g,$ and $fg.$ Also$,$ $f/g$ is continuous at each $\mathbf{X}_0$ in $S$ such that $g(\mathbf{X}_0)\ne0.$


For a vector-valued function $\mathbf{G},$ $$ \lim_{\mathbf{U}\to\mathbf{U}_0}\mathbf{G}(\mathbf{U})=\mathbf{L} $$ if and only if for each $\epsilon>0$ there is a $\delta>0$ such that $$ |\mathbf{G}(\mathbf{U})-\mathbf{L}|<\epsilon\mbox{\quad whenever\quad} 0<|\mathbf{U}-\mathbf{U}_0|<\delta\mbox{\quad and\quad}\mathbf{U}\in D_{\mathbf{G}}. $$ Similarly, $\mathbf{G}$ is continuous at $\mathbf{U}_0$ if and only if for each $\epsilon> 0$ there is a $\delta>0$ such that $$ |\mathbf{G}(\mathbf{U})-\mathbf{G}(\mathbf{U}_0)|<\epsilon \mbox{\quad whenever\quad}  |\mathbf{U}-\mathbf{U}_0|<\delta\mbox{\quad and\quad}\mathbf{U}\in D_{\mathbf{G}}. $$


Let $f$ be a real-valued function defined on a subset of $\R^n,$  and let the vector-valued function $\mathbf{G}=(g_1,g_2, \dots,g_n)$ be defined on a domain $D_\mathbf{G}$ in $\R^m.$ Let the set $$ T=\set{\mathbf{U}}{\mathbf{U}\in D_{\mathbf{G}}\mbox{\quad and \quad} \mathbf{G}(\mathbf{U})\in D_f} $$ $($Figure~\ref{figure:5.2.3}$)$,  be nonempty$,$ and define the real-valued composite function $$ h=f\circ\mathbf{G} $$ on $T$ by $$ h(\mathbf{U})=f(\mathbf{G}(\mathbf{U})),\quad \mathbf{U}\in T. $$ Now suppose that  $\mathbf{U}_0$ is in $T$ and is a limit point of $T,$ $\mathbf{G}$ is continuous at $\mathbf{U}_0,$ and $f$ is continuous at $\mathbf{X}_0=\mathbf{G}(\mathbf{U}_0).$ Then $h$ is continuous at $\mathbf{U}_0.$


If $f$ is continuous on a compact set $S$ in $\R^n,$ then $f$ is bounded on~$S.$


Let $f$ be continuous on a compact set $S$ in $\R^n$ and $$ \alpha=\inf_{\mathbf{X}\in S}f(\mathbf{X}),\quad\beta= \sup_{\mathbf{X}\in S}f(\mathbf{X}). $$ Then $$ f(\mathbf{X}_1)=\alpha\mbox{\quad and\quad} f(\mathbf{X}_2)=\beta $$ for some $\mathbf{X}_1$ and $\mathbf{X}_2$ in $S.$


Intermediate Value Theorem

Let $f$ be continuous on a region $S$ in $\R^n.$ Suppose that $\mathbf{A}$ and $\mathbf{B}$ are in $S$ and $$ f(\mathbf{A})<u<f(\mathbf{B}). $$ Then $f(\mathbf{C})=u$ for some $\mathbf{C}$ in $S.$


If $f$ is continuous on a compact set $S$ in $\R^n,$ then $f$ is uniformly continuous on $S.$


If $f_{x_i} (\mathbf{X})$ and $g_{x_i} (\mathbf{X})$ exist$,$ then \begin{eqnarray*} \frac{\partial (f+g)(\mathbf{X})}{\partial x_i}\ar=f_{x_i}(\mathbf{X})+ g_{x_i}(\mathbf{X}),\\ \frac{\partial (fg)(\mathbf{X})}{\partial x_i}\ar=f_{x_i}(\mathbf{X}) g(\mathbf{X})+f(\mathbf{X})g_{x_i} (\mathbf{X}), \end{eqnarray*} \newpage \noindent and$,$ if $g(\mathbf{X})\ne0,$ $$ \frac{\partial (f/g)(\mathbf{X})}{\partial x_i}=\frac{g(\mathbf{X})f_{x_i} (\mathbf{X})- f(\mathbf{X})g_{x_i}(\mathbf{X})}{[g(\mathbf{X})]^2}. $$


Suppose that  $f,$ $f_x,$ $f_y,$ and $f_{xy}$ exist on a neighborhood $N$ of $(x_0,y_0),$ and $f_{xy}$ is continuous at $(x_0,y_0).$ Then $f_{yx}(x_0,y_0)$ exists, and \begin{equation}\label{eq:5.3.5} f_{yx}(x_0,y_0)=f_{xy}(x_0,y_0). \end{equation}


Suppose that  $f$ and all its partial derivatives of order $\le r$ are continuous on an open subset $S$ of $\R^n.$  Then \begin{equation}\label{eq:5.3.11} f_{x_{i_1}x_{i_2}, \dots, x_{i_r}}(\mathbf{X})=f_{x_{j_1}x_{j_2}, \dots, x_{j_r}}(\mathbf{X}),\quad \mathbf{X}\in S, \end{equation} if each of the variables $x_1,$ $x_2,$ \dots$,$ $x_n$ appears the same number of times in $$ \{x_{i_1}, x_{i_2}, \dots,x_{i_r}\}\mbox{\quad and \quad} \{x_{j_1},x_{j_2}, \dots,x_{j_r}\}. $$  If this number is $r_k,$ we denote the common value of the two sides of $\eqref{eq:5.3.11}$ by \begin{equation}\label{eq:5.3.12} \frac{\partial^r f(\mathbf{X})}{\partial x^{r_1}_1\partial x^{r_2}_2\cdots \partial x^{r_n}_n}, \end{equation} it being understood that \begin{equation}\label{eq:5.3.13} 0\le r_k\le r,\quad 1\le k\le n, \end{equation} \begin{equation}\label{eq:5.3.14} r_1+r_2+\cdots+r_n=r, \end{equation} and$,$ if $r_k=0,$ we omit the symbol $\partial x_k^0$  from the ``denominator'' of $\eqref{eq:5.3.12}.$


If $f$ is differentiable at $\mathbf{X}_0=(x_{10},x_{20}, \dots,x_{n0}),$ then $f_{x_1}(\mathbf{X}_0),$ $f_{x_2}(\mathbf{X}_{0}),$ \dots$,$ $f_{x_n}(\mathbf{X}_0)$ exist and the constants  $m_1,$ $m_2,$ \dots$,$ $m_n$ in $\eqref{eq:5.3.16}$ are given by \begin{equation}\label{eq:5.3.18} m_i=f_{x_i}(\mathbf{X}_0),\quad 1\le i\le n; \end{equation} that is$,$ $$ \lim_{\mathbf{X}\to\mathbf{X}_0} \frac{f(\mathbf{X})-f(\mathbf{X}_0)- \dst{\sum^n_{i=1}}\, f_{x_i}(\mathbf{X}_0) (x_i-x_{i0})} { |\mathbf{X}-\mathbf{X}_0|}=0. $$


If $f$ is differentiable at $\mathbf{X}_0,$ then $f$ is continuous at $\mathbf{X}_0$.


$\mathbf{X}_0,$ then so are $f+g$ and $fg$. The same is true of $f/g$ if $g(\mathbf{X}_0)\ne0$. The differentials are given by \begin{eqnarray*} d_{\mathbf{X}_0}(f+g)\ar=d_{\mathbf{X}_0}f+d_{\mathbf{X}_0}g,\\ d_{\mathbf{X}_0}(fg)\ar=f(\mathbf{X}_0)d_{\mathbf{X}_0} g+g(\mathbf{X}_0) d_{\mathbf{X}_0}f,\\ \noalign{\hbox{and}} d_{\mathbf{X}_0}\left(\frac{f}{ g}\right)\ar=\frac{g(\mathbf{X}_0)d_{\mathbf{X}_0}f-f(\mathbf{X}_0) d_{\mathbf{X}_0}g}{[g(\mathbf{X}_0)]^2}. \end{eqnarray*}


If $f_{x_1},$ $f_{x_2},$ \dots$,$ $f_{x_n}$ exist on a neighborhood of $\mathbf{X}_0$ and are continuous at $\mathbf{X}_0,$ then $f$ is differentiable at $\mathbf{X}_0.$


Suppose that  $f$ is defined in a neighborhood of $\mathbf{X}_0$ in $\R^n$  and $f_{x_1}(\mathbf{X}_0),$ $f_{x_2}(\mathbf{X}_{0}),$  \dots$,$ $f_{x_n}(\mathbf{X}_{0})$  exist$.$ Let $\mathbf{X}_0$ be a local extreme point of $f.$ Then \begin{equation}\label{eq:5.3.42} f_{x_i}(\mathbf{X}_0)=0,\quad 1\le i\le n. \end{equation}


The Chain Rule

Suppose that  the real-valued function $f$ is differentiable at $\mathbf{X}_0$ in $\R^n,$ the vector-valued function $\mathbf{G} =(g_1,g_2, \dots,g_n)$ is differentiable at $\mathbf{U}_0$ in $\R^m,$ and $\mathbf{X}_{0}  = \mathbf{G}(\mathbf{U}_0).$ Then the real-valued composite function $h=f\circ\mathbf{G}$ defined by \begin{equation} \label{eq:5.4.3} h(\mathbf{U})=f(\mathbf{G}(\mathbf{U})) \end{equation} is differentiable at $\mathbf{U}_0,$ and \begin{equation} \label{eq:5.4.4} d_{\mathbf{U}_0}h=f_{x_1}(\mathbf{X}_0) d_{\mathbf{U}_0}g_1+f_{x_2} (\mathbf{X}_0) d_{\mathbf{U}_0}g_2+\cdots +f_{x_n} (\mathbf{X}_0) d_{\mathbf{U}_0}g_n. \end{equation}


Mean Value Theorem for Functions of $\mathbf n$ Variables

Let $f$ be continuous at $\mathbf{X}_1=(x_{11},x_{21}, \dots, x_{n1})$ and $\mathbf{X}_2=(x_{12},x_{22}, \dots,x_{n2})$ and differentiable on the line segment $L$ from $\mathbf{X}_1$ to $\mathbf{X}_2.$ Then \begin{equation} \label{eq:5.4.21} f(\mathbf{X}_2)-f(\mathbf{X}_1)=\sum_{i=1}^n f_{x_i} (\mathbf{X}_0)(x_{i2}-x_{i1})=(d_{\mathbf{X}_0}f)(\mathbf{X}_2 -\mathbf{X}_1) \end{equation} for some $\mathbf{X}_0$ on $L$ distinct from $\mathbf{X}_1$ and $\mathbf{X}_2$.


Taylor's Theorem for Functions of $\mathbf n$ Variables

Suppose \\that  $f$ and its partial derivatives of order $\le k$ are differentiable at $\mathbf{X}_0$ and $\mathbf{X}$ in $\R^n$ and on the line segment $L$ connecting them$.$ Then \begin{equation} \label{eq:5.4.25} f(\mathbf{X})=\sum_{r=0}^k\frac{1}{ r!} (d^{(r)}_{\mathbf{X}_0}f)  (\mathbf{X}-\mathbf{X})+\frac{1}{(k+1)!} (d^{(k+1)}_{\widetilde{\mathbf{X}}} f)(\mathbf{X}-{\mathbf{X}_0}) \end{equation} for some $\widetilde{\mathbf{X}}$  on $L$ distinct from $\mathbf{X}_0$ and $\mathbf{X}$.


Suppose that  $f$ and its partial derivatives of order $\le k-1$ are differentiable in a neighborhood $N$ of a point $\mathbf{X}_0$ in $\R^n$ and all $k$th-order partial derivatives of $f$ are continuous at $\mathbf{X}_0.$ Then \begin{equation} \label{eq:5.4.31} \lim_{\mathbf{X}\to\mathbf{X}_0} \frac{f(\mathbf{X})-T_k(\mathbf{X})}{ |\mathbf{X}-\mathbf{X}_0|^k}=0. \end{equation}


Suppose that  $f$ satisfies the hypotheses of Theorem~$\ref{thmtype:5.4.9}$ with $k\ge2,$ and  \begin{equation} \label{eq:5.4.38} d^{(r)}_{\mathbf{X}_0} f\equiv0\quad (1\le r\le k-1),\quad d^{(k)}_\mathbf{X_0} f\not\equiv0. \end{equation} Then \begin{alist} \item % (a) $\mathbf{X}_0$ is not a local extreme point of $f$ unless $d^{(k)}_{\mathbf{X}_0}f$ is semidefinite as a polynomial in $\mathbf{X}-\mathbf{X}_0.$ In particular$,$  $\mathbf{X}_0$ is not a local extreme point of $f$ if $k$ is odd$.$ \item % (b)  $\mathbf{X}_0$ is a local minimum point of $f$ if $d^{(k)}_{\mathbf{X}_0} f$ is positive definite$,$ or a local maximum point if $d^{(k)}_{\mathbf{X}_0}f$ is negative definite$.$ \item % (c)  If $d^{(k)}_{\mathbf{X}_0}f$ is semidefinite$,$ then $\mathbf{X}_0$ may be a local extreme point of $f,$ but it need not be$.$ \end{alist}


A transformation $\mathbf{L}: \R^n \to \R^m$ defined on all of $\R^n$ is linear if and only if \begin{equation}\label{eq:6.1.1} \mathbf{L}(\mathbf{X})=\left[\begin{array}{c} a_{11}x_1+a_{12}x_2+ \cdots+a_{1n}x_n\\a_{21}x_1+a_{22}x_2+\cdots+a_{2n}x_n\\ \vdots\\a_{m1}x_1+a_{m2}x_2+\cdots+a_{mn}x_n\end{array}\right], \end{equation} where the $a_{ij}$'s are constants$.$


If $\mathbf{A},$ $\mathbf{B},$ and $\mathbf{C}$ are $m\times n$ matrices$,$ then $$ (\mathbf{A}+\mathbf{B})+\mathbf{C}=\mathbf{A}+(\mathbf{B} +\mathbf{C}). $$


If $\mathbf{A}$ and $\mathbf{B}$ are $m\times n$ matrices and $r$ and $s$ are real numbers$,$ then \part{a} $r(s\mathbf{A}) =(rs)\mathbf{A};$ \part{b} $(r+s)\mathbf{A}=r\mathbf{A}+s\mathbf{A};$ \part{c} $r(\mathbf{A}+\mathbf{B})=r\mathbf{A}+r\mathbf{B}.$


If $\mathbf{A},$ $\mathbf{B},$ and $\mathbf{C}$ are $m\times p,$ $p\times q,$ and $q\times n$ matrices$,$ respectively$,$ then $(\mathbf{AB})\mathbf{C}=\mathbf{A}(\mathbf{BC}).$


\begin{alist} \item % (a) If we regard the vector $$ \mathbf{X}=\left[\begin{array}{c} x_1\\ x_2\\\vdots\\ x_n\end{array}\right] $$ as an $n\times 1$ matrix$,$ then the linear transformation $\eqref{eq:6.1.1}$ can be written as $$ \mathbf{L}(\mathbf{X})=\mathbf{AX}. $$ \newpage \noindent \item % (b) If $\mathbf{L}_1$ and $\mathbf{L}_2$ are linear transformations from $\R^n$ to $\R^m$ with matrices $\mathbf{A}_1$ and $\mathbf{A}_{2}$ respectively$,$ then $c_1\mathbf{L}_1+c_2\mathbf{L}_2$ is the linear transformation from $\R^n$ to $\R^m$ with matrix $c_1\mathbf{A}_1+c_2\mathbf{A}_{2}.$ \item % (c) If $\mathbf{L}_1: \R^n\to \R^p$ and $\mathbf{L}_2: \R^p\to \R^m$ are linear transformations with matrices $\mathbf{A}_1$ and $\mathbf{A}_2,$ respectively$,$ then the composite function $\mathbf{L}_3=\mathbf{L}_2\circ\mathbf{L}_1,$ defined by $$ \mathbf{L}_3(\mathbf{X})=\mathbf{L}_2(\mathbf{L}_1(\mathbf{X})), $$ is the linear transformation from $\R^n$ to $\R^m$ with matrix $\mathbf{A}_2\mathbf{A}_1.$ \end{alist}


If $\mathbf{A}$ and $\mathbf{B}$ are $n\times n$  matrices$,$ then $$ \det(\mathbf{A}\mathbf{B})=\det(\mathbf{A})\det(\mathbf{B}). $$


Let $\mathbf{A}$ be an $n\times n$ matrix$.$ \begin{alist} \item % (a) The sum of the products of the entries of a  row of $\mathbf{A}$ and their cofactors equals $\det(\mathbf{A}),$ while the  sum of the products of the entries of a  row of $\mathbf{A}$ and the cofactors of the entries of a different row equals zero$;$ that is$,$ \begin{equation} \label{eq:6.1.8} \sum^n_{k=1} a_{ik}c_{jk}=\left\{\casespace\begin{array}{ll}\det(\mathbf{A}),&i=j,\\  0,&i\ne j.\end{array}\right. \end{equation} \item % (b) The sum of the products of the entries of a  column of $\mathbf{A}$ and their cofactors equals $\det(\mathbf{A}),$ while the  sum of the products of the entries of a  column of $\mathbf{A}$ and the cofactors of the entries of a different column equals zero$;$ that is$,$ \begin{equation} \label{eq:6.1.9} \sum^n_{k=1} c_{ki}a_{kj}=\left\{\casespace\begin{array}{ll} \det(\mathbf{A}), &i=j,\\  0,&i\ne j.\end{array}\right. \end{equation} \end{alist}


Let $\mathbf{A}$ be an $n\times n$ matrix$.$ If $\det(\mathbf{A})=0,$ then $\mathbf{A}$ is singular$.$ If $\det(\mathbf{A})\ne0,$ then $\mathbf{A}$ is nonsingular$,$ and $\mathbf{A}$ has the unique inverse \begin{equation} \label{eq:6.1.10} \mathbf{A}^{-1}=\frac{1}{\det(\mathbf{A})}\adj(\mathbf{A}). \end{equation}


The system $\eqref{eq:6.1.11}$ has a solution $\mathbf{X}$ for any given $\mathbf{Y}$ if and only if $\mathbf{A}$ is nonsingular$.$ In this case$,$ the solution is unique and is given by $\mathbf{X}=\mathbf{A}^{-1}\mathbf{Y}$.


If $\mathbf{A}=[a_{ij}]$ is nonsingular$,$ then the solution of  the system \begin{eqnarray*} a_{11}x_1+a_{12}x_2+\cdots+a_{1n}x_n\ar=y_1\\ a_{21}x_1+a_{22}x_2+\cdots+a_{2n}x_n\ar=y_2\\ &\vdots& \\ a_{n1}x_1+a_{n2}x_2+\cdots+a_{nn}x_n\ar=y_n \end{eqnarray*} $($or$,$ in matrix form$,$ $\mathbf{AX}=\mathbf{Y}$$)$ is given by $$ x_i=\frac{D_i}{\det(\mathbf{A})},\quad 1\le i\le n, $$ where $D_i$ is the determinant of the matrix obtained by replacing the $i$th column of $\mathbf{A}$ with $\mathbf{Y};$ thus$,$ $$ D_1=\left|\begin{array}{cccc} y_1&a_{12}&\cdots&a_{1n}\\ y_2&a_{22}&\dots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ y_n&a_{n2}&\cdots&a_{nn}\end{array}\right|,\quad D_2=\left|\begin{array}{ccccc} a_{11}&y_1&a_{13}&\cdots&a_{1n}\\ a_{21}&y_2&a_{23}&\cdots&a_{2n}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ a_{n1}&y_n&a_{n3}&\cdots&a_{nn}\end{array}\right|,\quad\cdots, $$ $$ D_n=\left|\begin{array}{cccc}  a_{11}&\cdots&a_{1,n-1}&y_1\\ a_{21}&\cdots&a_{2,n-1}&y_2\\ \vdots&\vdots&\ddots&\vdots\\ a_{n1}&\cdots&a_{n,n-1}&y_n\end{array}\right|. $$


The homogeneous system  $\eqref{eq:6.1.12}$ of $n$ equations in $n$ unknowns has a nontrivial solution if and only if $\det(\mathbf{A})=0.$


If $A_1,$ $A_2,$ \dots$,$ $A_k$ are nonsingular $n\times n$ matrices$,$ then so is $A_1A_2\cdots A_k,$ and $$ (A_1A_2\cdots A_k)^{-1}=A_k^{-1}A_{k-1}^{-1}\cdots A_1^{-1}. $$


Suppose that $\mathbf{X}_0$ is in$,$ and a limit point of$,$ the domain of $\mathbf{F}: \R^n\to\R^m.$ Then $\mathbf{F}$ is continuous at $\mathbf{X}_0$ if and only if for each $\epsilon>0$ there is a $\delta>0$ such that \begin{equation}\label{eq:6.2.1} |\mathbf{F}(\mathbf{X})-\mathbf{F}(\mathbf{X}_0)|<\epsilon \mbox{\quad if \quad} |\mathbf{X}-\mathbf{X}_0|<\delta \mbox{\quad and \quad} \mathbf{X}\in D_\mathbf{F}. \end{equation}


A transformation $\mathbf{F}=(f_1,f_2, \dots,f_m)$ defined in a neighborhood of $\mathbf{X}_0\in\R^n$  is differentiable at $\mathbf{X}_0$ if and only if there is a constant $m\times n$ matrix $\mathbf{A}$ such that \begin{equation}\label{eq:6.2.2} \lim_{\mathbf{X}\to\mathbf{X}_0} \frac{ \mathbf{F}(\mathbf{X})-\mathbf{F}(\mathbf{X}_0)-\mathbf{A} (\mathbf{X}-\mathbf{X}_0)} {|\mathbf{X}-\mathbf{X}_0|}=\mathbf{0}.  \end{equation} If $\eqref{eq:6.2.2}$ holds$,$ then $\mathbf{A}$ is given uniquely by \begin{equation}\label{eq:6.2.3} \mathbf{A}=\left[\frac{\partial f_i(\mathbf{X}_0)}{\partial x_j}\right]= \left[\begin{array}{cccc}\dst{\frac{\partial f_1(\mathbf{X}_0)}{\partial x_1}}& \dst{\frac{\partial f_1(\mathbf{X}_0)}{\partial x_2}}&\cdots& \dst{\frac{\partial f_1(\mathbf{X}_0)}{\partial x_n}}\\ [3\jot] \dst{\frac{\partial f_2(\mathbf{X}_0)}{\partial x_1}}& \dst{\frac{\partial f_2(\mathbf{X}_0)}{\partial x_2}}& \cdots&\dst{\frac{\partial f_2(\mathbf{X}_0)}{\partial x_n}}\\ \vdots&\vdots&\ddots&\vdots\\ \dst{\frac{\partial f_m(\mathbf{X}_0)}{\partial x_1}}& \dst{\frac{\partial f_m(\mathbf{X}_0)}{\partial x _2}}& \cdots&\dst{\frac{\partial f_m(\mathbf{X}_0)}{\partial x_n}} \end{array}\right]. \end{equation}


If $\mathbf{F}: \R^n\to\R^m$ is differentiable at $\mathbf{X}_0,$ then $\mathbf{F}$ is continuous at~$\mathbf{X}_0.$


Let  $\mathbf{F}=(f_1,f_2, \dots,f_m):\R^n\to\R^m,$ and suppose that the partial derivatives \begin{equation}\label{eq:6.2.7} \frac{\partial f_i}{\partial x_j},\quad 1\le i\le m,\quad 1\le j\le n, \end{equation} exist on a neighborhood of $\mathbf{X}_0$ and are continuous at $\mathbf{X}_0.$ Then $\mathbf{F}$ is differentiable at $\mathbf{X}_0.$


Suppose that $\mathbf{F}: \R^n\to\R^m$ is differentiable at $\mathbf{X}_0,$ $\mathbf{G}:\R^k\to\R^n$ is differentiable at $\mathbf{U}_0,$ and $\mathbf{X}_0=\mathbf{G}(\mathbf{U}_0).$ Then the composite function $\mathbf{H}=\mathbf{F}\circ\mathbf{G}:\R^k\to\R^m,$ defined by $$ \mathbf{H}(\mathbf{U})=\mathbf{F}(\mathbf{G}(\mathbf{U})), $$ is differentiable at $\mathbf{U}_0.$ Moreover$,$ \begin{equation}\label{eq:6.2.22} \mathbf{H}'(\mathbf{U}_0)=\mathbf{F}'(\mathbf{G}(\mathbf{U}_0)) \mathbf{G}'(\mathbf{U}_0) \end{equation} and \begin{equation}\label{eq:6.2.23} d_{\mathbf{U}_0}\mathbf{H}=d_{\mathbf{X}_0}\mathbf{F}\circ d_{\mathbf{U}_0}\mathbf{G}, \end{equation} where $\circ$ denotes composition$.$


The linear transformation $$ \mathbf{U}=\mathbf{L}(\mathbf{X})=\mathbf{A}\mathbf{X}\quad  (\R^n\to \R^n) $$ is invertible if and only if $\mathbf{A}$ is nonsingular$,$ in which case $R(\mathbf{L})= \R^n$ and $$ \mathbf{L}^{-1}(\mathbf{U})=\mathbf{A}^{-1}\mathbf{U}. $$


Suppose that $\mathbf{F}: \R^n\to \R^n$ is regular on an open set $S,$ and let $\mathbf{G}=\mathbf{F}^{-1}_S.$ Then $\mathbf{F}(S)$ is open$,$ $\mathbf{G}$ is continuously differentiable on $\mathbf{F}(S),$ and $$ \mathbf{G}'(\mathbf{U})=(\mathbf{F}'(\mathbf{X}))^{-1}, \mbox{\quad where\quad}\mathbf{U}=\mathbf{F}(\mathbf{X}). $$ Moreover$,$ since $\mathbf{G}$ is one-to-one on $\mathbf{F}(S),$  $\mathbf{G}$ is regular on $\mathbf{F}(S).$


The Inverse Function Theorem

Let $\mathbf{F}: \R^n\to \R^n$ be continuously differentiable on an open set $S,$ and suppose that $J\mathbf{F}(\mathbf{X})\ne0$ on $S.$ Then$,$ if $\mathbf{X}_0\in S,$ there is an open neighborhood $N$ of $\mathbf{X}_0$ on which $\mathbf{F}$ is regular$.$ Moreover$,$ $\mathbf{F}(N)$ is open and $\mathbf{G}= \mathbf{F}^{-1}_N$ is continuously differentiable on $\mathbf{F}(N),$ with $$ \mathbf{G}'(\mathbf{U})=\left[\mathbf{F}'(\mathbf{X})\right]^{-1}\quad \mbox{ $($where $\mathbf{U}=\mathbf{F}(\mathbf{X})$$)$},\quad \mathbf{U}\in\mathbf{F}(N). $$


The Implicit Function Theorem

Suppose that  $\mathbf{F}:\R^{n+m}\to \R^m$ is continuously differentiable on an open set $S$ of $\R^{n+m}$ containing $(\mathbf{X}_0,\mathbf{U}_0).$ Let $\mathbf{F}(\mathbf{X}_0,\mathbf{U}_0)=\mathbf{0},$ and suppose that  $\mathbf{F}_\mathbf{U}(\mathbf{X}_0,\mathbf{U}_0)$ is nonsingular$.$ Then there is a neighborhood $M$ of  $(\mathbf{X}_0,\mathbf{U}_{0}),$  contained in $S,$ on which  $\mathbf{F}_\mathbf{U}(\mathbf{X},\mathbf{U})$  is nonsingular  and a neighborhood $N$ of $\mathbf{X}_0$ in $\R^n$ on which a unique  continuously differentiable  transformation $\mathbf{G}: \R^n\to \R^m$ is defined$,$ such that $\mathbf{G}(\mathbf{X}_0)=\mathbf{U}_0$ and \begin{equation} \label{eq:6.4.6} (\mathbf{ X},\mathbf{G}(\mathbf{X}))\in M\mbox{\quad and \quad} \mathbf{F}(\mathbf{X},\mathbf{G}(\mathbf{X}))=0\mbox{\quad  if}\quad\mathbf{X}\in N. \end{equation} Moreover$,$ \begin{equation} \label{eq:6.4.7} \mathbf{G}'(\mathbf{X})=-[\mathbf{F}_\mathbf{U}(\mathbf{X},\mathbf{G}(\mathbf{X}))]^{-1} \mathbf{F}_\mathbf{X}(\mathbf{X},\mathbf{G}(\mathbf{X})),\quad \mathbf{X}\in N. \end{equation}


If $f$ is unbounded on the nondegenerate rectangle $R$ in $\R^n,$ then $f$ is not integrable on $R.$


Let $f$ be bounded on a rectangle  $R$  and let $\mathbf{P}$ be a partition of $R.$  Then \begin{alist} \item % (a)  The upper sum $S(\mathbf{P})$ of $f$ over $\mathbf{P}$ is the supremum of the set of all Riemann sums of $f$ over $\mathbf{P}.$ \item % (b)  The lower sum $s(\mathbf{P})$ of $f$ over $\mathbf{P}$ is the infimum  of the set of all Riemann sums of $f$ over $\mathbf{P}.$ \end{alist}


If $f$ is bounded on a rectangle $R,$ then $$ \underline{\int_R}\, f(\mathbf{X})\,d\mathbf{X} \le\overline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}. $$


If $f$ is integrable on a rectangle $R,$ then $$ \underline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}= \overline{\int_R}\, f(\mathbf{X})\,d\mathbf{X} =\int_R f(\mathbf{X})\,d\mathbf{X}. $$


If $f$ is bounded on a rectangle $R$ and \vspace{2pt} $$ \underline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}= \overline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}=L, $$ \vspace{2pt} then $f$ is integrable on $R,$ and \vspace{2pt} $$ \int_R f(\mathbf{X})\,d\mathbf{X}=L. $$


A bounded function $f$ is integrable on a rectangle $R$ if and only if $$ \underline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}=\overline{\int_R}\, f(\mathbf{X})\, d\mathbf{X}. $$


If $f$ is bounded on a rectangle $R,$ then $f$ is integrable on $R$ if and only if for every $\epsilon>0$ there is a partition ${\bf P}$ of $R$ such that $$ S({\bf P})-s({\bf P})<\epsilon. $$


If $f$ is continuous on a rectangle $R$ in $\R^n,$ then $f$ is integrable on~$R.$


Suppose that $f$ is bounded on a rectangle \begin{equation}\label{eq:7.1.30} R=[a_1,b_1]\times [a_2,b_2]\times\cdots\times [a_n,b_n] \end{equation} and continuous except on a subset $E$ of $R$ with zero content$.$ Then $f$ is integrable on $R.$


Suppose that $f$ is bounded on a bounded set $S$ and continuous except on a subset $E$ of $S$ with zero content. Suppose also that $\partial S$ has zero content$.$ Then $f$ is integrable on $S.$


A differentiable surface in $\R^n$ has zero content$.$


Suppose that $S$ is a bounded set in $\R^n,$ with boundary consisting of a finite number of differentiable surfaces$.$ Let $f$ be bounded on $S$ and continuous except on a set of zero content. Then $f$ is integrable on $S.$


If $f$ and $g$ are integrable on $S,$ then so is $f+g,$ and $$ \int_S(f+g)(\mathbf{X})\,d\mathbf{X}=\int_S f(\mathbf{X})\,d\mathbf{X}+ \int_S g(\mathbf{X})\,d\mathbf{X}. $$


If $f$ is integrable on $S$ and $c$ is a constant$,$ then $cf$ is integrable on $S,$ and $$ \int_S(cf)(\mathbf{X})\,d\mathbf{X}=c\int_S f(\mathbf{X})\,d\mathbf{X}. $$


If $f$ and $g$ are integrable on $S$ and $f(\mathbf{X})\le g(\mathbf{X})$ for $\mathbf{X}$ in $S,$ then $$ \int_S f(\mathbf{X})\,d\mathbf{X}\le\int_S g(\mathbf{X})\,d\mathbf{X}. $$


If $f$ is integrable on $S,$ then so is $|f|,$ and $$ \left|\int_S f(\mathbf{X})\,d\mathbf{X}\right|\le\int_S |f(\mathbf{X})|\,d\mathbf{X}. $$


If $f$ and $g$ are integrable on $S,$ then so is the product $fg.$


Suppose that $u$ is continuous and $v$ is integrable and nonnegative on a rectangle $R.$ Then $$ \int_R u(\mathbf{X})v(\mathbf{X})\,d\mathbf{X}= u(\mathbf{X}_0)\int_R v(\mathbf{X})\,d\mathbf{X} $$ for some $\mathbf{X}_0$ in $R.$


If $f$ is integrable on disjoint sets $S_1$ and $S_2,$ then $f$ is integrable on $S_1\cup S_2,$ and \begin{equation}\label{eq:7.1.39} \int_{S_1\cup S_2} f(\mathbf{X})\,d\mathbf{X}= \int_{S_1} f(\mathbf{X})\,d\mathbf{X}+ \int_{S_2} f(\mathbf{X})\,d\mathbf{X}. \end{equation}


$R= [a,b]\times [c,d]$ and $$  F(y)=\int_a^b f(x,y)\,dx $$ exists for each $y$ in $[c,d].$ Then $F$ is integrable on $[c,d],$ and \begin{equation}\label{eq:7.2.1} \int_c^d F(y)\,dy=\int_R f(x,y)\,d(x,y); \end{equation} that is$,$ \begin{equation}\label{eq:7.2.2} \int_c^d dy\int_a^b f(x,y)\,dx=\int_R f(x,y)\,d(x,y). \end{equation}


Let $I_1,$ $I_2,$ \dots$,$ $I_n$ be closed intervals and suppose that  $f$ is integrable on $R=I_1\times I_2\times\cdots\times I_n.$ Suppose that  there is an integer $p$ in $\{1,2, \dots,n-1\}$ such that $$ F_p(x_{p+1},x_{p+2}, \dots,x_n)=\int_{I_1\times I_2\times\cdots\times I_p} f(x_1,x_2, \dots,x_n)\,d(x_1,x_2, \dots,x_p) $$ exists for each $(x_{p+1},x_{p+2}, \dots,x_n)$ in $I_{p+1}\times I_{p+2}\times\cdots\times I_n.$ Then $$ \int_{I_{p+1}\times I_{p+2}\times\cdots\times I_n} F_p(x_{p+1}, x_{p+2}, \dots,x_n)\,d(x_{p+1},x_{p+2}, \dots,x_n) $$ exists and equals $\int_R f(\mathbf{X})\,d\mathbf{X}$.


Let $I_j=[a_j,b_j],$ $1\le j\le n$, and suppose that $f$ is integrable on $R=I_1\times I_2 \times\cdots\times I_n.$ Suppose also that  the integrals $$ F_p(x_{p+1}, \dots,x_n)=\int_{I_1\times I_2\cdots\times I_p} f(\mathbf{X}) \,d(x_1,x_2, \dots,x_p),\quad1\le p\le n-1, $$ exist for all $$ (x_{p+1}, \dots,x_n)\mbox{\quad in\quad} I_{p+1}\times\cdots\times I_n. $$ Then the iterated integral $$ \int^{b_n}_{a_n} dx_n\int^{b_{n-1}}_{a_{n-1}} dx_{n-1}\cdots \int^{b_2}_{a_2} dx_2\int^{b_1}_{a_1} f(\mathbf{X})\,dx_1 $$ exists and equals $\int_R f(\mathbf{X})\,d\mathbf{X}.$


If $f$ is continuous on $$ R=[a_1,b_1]\times [a_2,b_2]\times\cdots\times [a_n,b_n], $$ then $\int_R f(\mathbf{X})\,d\mathbf{X}$ can be evaluated by iterated integrals in any of the $n!$ ways indicated in $\eqref{eq:7.2.16}.$


If $f$ is integrable on the set $S$ in $\eqref{eq:7.2.17}$ and the integral $\eqref{eq:7.2.19}$ exists for $c\le y\le d,$ then \begin{equation}\label{eq:7.2.20} \int_S f(x,y) \,d(x,y)=\int_c^d dy\int^{v(y)}_{u(y)} f(x,y)\,dx. \end{equation}


Suppose that  $f$ is integrable on $$ S=\set{(x,y,z)}{u_1(y,z)\le x\le v_1(y,z),\ u_2(z)\le y\le v_2(z),\ c\le z\le d}, $$ and let $$ S(z)=\set{(x,y)}{u_1(y,z)\le x\le v_1(y,z),\ u_2(z)\le y\le v_2(z)} $$ for each $z$ in $[c,d].$  Then $$ \int_S f(x,y,z)\,d(x,y,z)=\int_c^d dz\int^{v_2(z)}_{u_2(z)} dy \int^{v_1(y,z)}_{u_1(y,z)} f(x,y,z)\,dx, $$ provided that $$ \int^{v_1(y,z)}_{u_1(y,z)} f(x,y,z)\,dx $$ exists for all $(y,z)$ such that $$ c\le z\le d\mbox{\quad and\quad} u_2(z)\le y\le v_2(z), $$ and $$ \int_{S(z)} f(x,y,z)\,d(x,y) $$ exists for all $z$ in $[c,d].$


A bounded set $S$ is Jordan measurable if and only if the boundary of $S$ has zero content$.$


Suppose that  $\mathbf{G}:\R^n\to \R^n$ is regular on a compact Jordan measurable set $S.$ Then $\mathbf{G}(S)$ is compact and Jordan measurable$.$


If $S$ is a compact Jordan measurable subset  of $\R^n$ and $\mathbf{L}:\R^n\to \R^n$ is the invertible linear transformation $\mathbf{X}=\mathbf{L}(\mathbf{Y})=\mathbf{AY},$ then \begin{equation}\label{eq:7.3.14} V(\mathbf{L}(S))=|\det(\mathbf{A})| V(S). \end{equation}


\E^n\to \R^n$ is regular on a compact Jordan measurable set $S$ and $f$ is continuous on $\mathbf{G}(S).$ Then \begin{equation}\label{eq:7.3.28} \int_{\mathbf{G}(S)} f(\mathbf{X})\,d\mathbf{X}= \int_S f(\mathbf{G}(\mathbf{Y})) |J\mathbf{G}(\mathbf{Y})|\,d\mathbf{Y}. \end{equation}


Suppose that  $\mathbf{G}: \E^n\to \R^n$ is continuously differentiable on a bounded open set $N$ containing the compact Jordan measurable set $S,$ and regular on $S^0.$ Suppose   also that $\mathbf{G}(S)$ is Jordan measurable$,$ $f$ is continuous on $\mathbf{G}(S),$ and $G(C)$ is Jordan measurable for every cube $C\subset N$. Then \begin{equation}\label{eq:7.3.50} \int_{\mathbf{G}(S)} f(\mathbf{X})\,d\mathbf{X}= \int_S f(\mathbf{G}(\mathbf{Y})) |J\mathbf{G}(\mathbf{Y})|\,d\mathbf{Y}. \end{equation}


If $(A,N)$ is a normed vector space$,$ then \begin{equation} \label{eq:8.1.1} \rho(x,y)=N(x-y) \end{equation} is a metric on $A.$


If $x$ and $y$ are vectors in  a normed vector space $(A,N),$ then \begin{equation} \label{eq:8.1.2} |N(x)-N(y)|\le N(x-y). \end{equation}


If $\mathbf{X}\in\R^n$ and $p_2>p_1\ge1,$ then \begin{equation} \label{eq:8.1.12} \|\mathbf{X}\|_{p_2}\le\|\mathbf{X}\|_{p_1}; \end{equation} moreover, \begin{equation} \label{eq:8.1.13} \lim_{p\to\infty}\|\mathbf{X}\|_{p}=\max\set{|x_i|}{1\le i\le n}. \end{equation}


\begin{alist} \item % (a)   The union of open sets is open. \item % (b)  The intersection of closed sets is closed. \end{alist}


\begin{alist} \item  % (a) The limit of a convergent sequence is unique$.$ \item % (b) If $\lim_{n\to\infty}u_n=u,$  then every subsequence of $\{u_n\}$  converges to $u.$ \end{alist}


If a sequence $\{u_n\}$ in a metric space $(A,\rho)$ is convergent$,$ then it is a Cauchy sequence.


The Principle of Nested Sets

A metric space $(A,\rho)$ is complete if and only if every nested sequence $\{T_n\}$ of nonempty closed subsets of $A$ such that  $\lim_{n\to\infty}d(T_n)=0$ has a nonempty intersection$.$


If $\rho$ and $\sigma$ are equivalent  metrics on a set $A,$ then  $(A,\rho)$ and $(A,\sigma)$ have the same open sets.


Any two norms  $N_1$ and $N_2$ on $\R^n$ induce equivalent metrics on~$\R^n.$


Suppose that $\rho$ and $\sigma$ are equivalent metrics on $A.$ Then \begin{alist} \item % (a) A sequence $\{u_n\}$ converges to $u$ in $(A,\rho)$ if and only if it converges to $u$ in~$(A,\sigma).$ \item % (a) A sequence $\{u_n\}$ is a Cauchy sequence in  $(A,\rho)$ if and only if it is a Cauchy sequence in $(A,\sigma).$ \item % (b) $(A,\rho)$ is complete if and only if $(A,\sigma)$ is complete$.$ \end{alist}


An infinite subset $T$ of $A$ is compact if and only if every infinite subset of $T$ has a limit point in $T.$


A subset $T$ of a metric $A$ is compact if and only if every infinite sequence $\{t_n\}$ of members of  $T$  has a subsequence that converges to a member of $T.$


If  $T$ is  compact$,$  then every Cauchy sequence $\{t_n\}_{n=1}^\infty$ in $T$ converges to a limit in $T.$


If $T$ is compact$,$ then $T$ is closed and bounded.


If $T$  is compact$,$ then $T$ is totally bounded.


If $(A,\rho)$ is complete and $T$  is closed and totally bounded$,$ then $T$ is compact.


A nonempty subset $T$ of $C[a,b]$ is compact if and only if it is closed$,$ uniformly bounded$,$ and equicontinuous.


Suppose that  ${\mathcal F}$ is an infinite uniformly bounded and equicontinuous family of functions on $[a,b].$  Then there is a sequence $\{f_n\}$ in ${\mathcal F}$ that converges uniformly to a continuous function  on $[a,b].$


Suppose that $\widehat u\in\overline D_f.$ Then \begin{equation} \label{eq:8.3.3} \lim_{u\to \widehat u}f(u)=\widehat v \end{equation} if and only if \begin{equation} \label{eq:8.3.4} \lim_{n\to\infty}f(u_n)=\widehat v \end{equation} for every sequence $\{u_n\}$ in $D_f$ such that \begin{equation} \label{eq:8.3.5} \lim_{n\to\infty}u_n=\widehat u. \end{equation}


A function $f$ is continuous at $\widehat u$ if and only if $$ \lim_{u\to\widehat u} f(u)=f(\widehat u). $$


A function $f$ is continuous at $\widehat u$ if and only if $$ \lim_{n\to\infty} f(u_n)=f(\widehat u) $$ whenever $\{u_n\}$ is a sequence in $D_f$  that converges to $\widehat u$.


If $f$ is continuous on a compact set $T,$ then $f(T)$ is compact.


If $f$ is continuous on a compact set $T,$ then $f$ is uniformly continuous on $T$.


Contraction Mapping Theorem

If $f$ is a contraction of a complete metric space $(A,\rho),$ then the equation \begin{equation} \label{eq:8.3.8} f(u)=u \end{equation} has a unique solution$.$


If $f$ is differentiable at $x_0,$ then \begin{equation}\label{eq:2.3.3} f(x)=f(x_0)+[f'(x_0)+E(x)](x-x_0), \end{equation} where $E$ is defined on a neighborhood of $x_0$ and $$ \lim_{x\to x_0} E(x)=E(x_0)=0. $$


If $f^{(n)}(x_0)$ exists$,$ then \begin{equation}\label{eq:2.5.7} f(x)=\sum_{r=0}^n\frac{f^{(r)}(x_0)}{ r!} (x-x_0)^r+E_n(x)(x-x_0)^n, \end{equation} where $$ \lim_{x\to x_0} E_n(x)=E_n(x_0)=0. $$


Suppose that \begin{equation} \label{eq:3.2.1} |f(x)|\le M,\quad a\le x\le b, \end{equation} and let $P'$ be a partition of $[a,b]$ obtained by adding $r$ points to a partition $P=\{x_0,x_1, \dots,x_n\}$ of $[a,b].$  Then \begin{eqnarray} S(P)\ge S(P')\ar\ge S(P)-2Mr\|P\|\label{eq:3.2.2}\\ \arraytext{and}\nonumber\\ s(P)\le s(P')\ar\le s(P)+2Mr\|P\|\label{eq:3.2.3}. \end{eqnarray}


If $f$ is bounded on $[a,b]$ and  $\epsilon>0,$   there is a $\delta>0$ such that \begin{equation} \label{eq:3.2.12} \overline{\int_a^b}f(x)\,dx\le S(P)<\overline{\int_a^b}f(x)\,dx+\epsilon \end{equation} and $$ \underline{\int_a^b} f(x)\,dx\ge s(P)>\underline{\int_a^b} f(x)\,dx-\epsilon $$ if $\|P\|<\delta$.


If $w_f(x)<\epsilon$ for $a\le x \le b,$ then there is a $\delta>0$ such that $W_f[a_1,b_1]\le\epsilon,$ provided that $[a_1,b_1]\subset [a,b]$ and $b_1-a_1<\delta.$


Let $f$ be bounded on $[a,b]$ and define $$ E_\rho=\set{x\in [a,b]}{w_f(x)\ge\rho}. $$ Then $E_\rho$ is closed$,$ and  $f$ is integrable on $[a,b]$ if and only if for every pair of positive numbers $\rho$ and $\delta,$ $E_\rho$ can be covered by finitely many open intervals $I_1,$ $I_2, $\dots$,$ $I_p$ such that \begin{equation} \label{eq:3.5.3} \sum_{j=1}^p L(I_j)<\delta. \end{equation}


Suppose that for $n$ sufficiently large  $($that is$,$ for $n \ge\mbox{some integer }N$$)$  the terms of $\sum_{n=k}^\infty a_n$ satisfy  some condition that implies convergence of an infinite series$.$ Then $\sum_{n=k}^\infty a_n$ converges$.$ Similarly, suppose that for $n$  sufficiently large  the terms $\sum_{n=k}^\infty a_n$ satisfy  some condition that implies divergence of an infinite series$.$ Then $\sum_{n=k}^\infty a_n$ diverges$.$


If $g$ and $h$ are defined on $S,$ then \begin{eqnarray*} \|g+h\|_S\ar\le\|g\|_S+\|h\|_S\\ \arraytext{and}\\ \|gh\|_S\ar\le\|g\|_S\|h\|_S. \end{eqnarray*} Moroever$,$ if either $g$ or $h$ is bounded on $S,$ then $$ \|g-h\|_S\ge\left|\|g\|_S-\|h\|_S\|\right|. $$


If $\mathbf{X}$ and $\mathbf{Y}$ are any two vectors in $\R^n,$ then \begin{equation} \label{eq:5.1.3} |\mathbf{X}\cdot\mathbf{Y}|\le |\mathbf{X}|\,|\mathbf{Y}|, \end{equation} with equality  if and only if one of the vectors is a scalar multiple of the other$.$


If $\mathbf{X}_1$ and $\mathbf{X}_2$ are in $S_r(\mathbf{X}_0)$ for some $r>0$, then so is every point on the line segment from $\mathbf{X}_1$ to $\mathbf{X}_2.$


If $f$ is differentiable at $\mathbf{X}_0,$ then $$ f(\mathbf{X})-f(\mathbf{X}_0)=(d_{\mathbf{X}_0}f)(\mathbf{X}-\mathbf{X}_0) +E(\mathbf{X})|\mathbf{X}-\mathbf{X}_0|, $$ where $E$  is defined in a neighborhood of $\mathbf{X}_0$ and $$ \lim_{\mathbf{X}\to\mathbf{X}_0}E(\mathbf{X})=E(\mathbf{X}_0)=0. $$


Suppose that  $\mathbf{G}=(g_1,g_2, \dots,g_n)$ is differentiable at $$  \mathbf{U}_0=(u_{10}, u_{20}, \dots,u_{m0}), $$  and  define $$ M=\left(\sum_{i=1}^n\sum_{j=1}^m\left(\frac{\partial g_i(\mathbf{U}_0} {\partial u_j}\right)^2\right)^{1/2}. $$ Then$,$ if $\epsilon>0,$ there is a $\delta>0$ such that $$ \frac{|\mathbf{G}(\mathbf{U})-\mathbf{G}(\mathbf{U}_0)|} {|\mathbf{U}-\mathbf{U}_{0}|} <M+\epsilon\mbox{\quad if \quad}0<|\mathbf{U}-\mathbf{U}_0|<\delta. $$


Suppose that $\mathbf{F}:\R^n\to\R^m$ is continuously differentiable on a neighborhood $N$ of $\mathbf{X}_0.$ Then$,$ for every $\epsilon>0,$ there is a $\delta>0$ such that \begin{equation}\label{eq:6.2.8} |\mathbf{F}(\mathbf{X})-\mathbf{F}(\mathbf{Y})|< (\|\mathbf{F}'(\mathbf{X}_{0})\| +\epsilon) |\mathbf{X}-\mathbf{Y}| \mbox{\quad if\quad}\mathbf{A},\mathbf{Y}\in B_\delta (\mathbf{X}_0). \end{equation}


Suppose that $\mathbf{F}:\R^n\to\R^n$ is continuously differentiable on a neighborhood of $\mathbf{X}_0$  and $\mathbf{F}'(\mathbf{X}_0)$ is nonsingular$.$ Let \begin{equation}\label{eq:6.2.14} r=\frac{1}{\|(\mathbf{F}'(\mathbf{X}_0))^{-1}\|}. \end{equation} Then$,$ for every $\epsilon>0,$ there is a $\delta>0$ such that \begin{equation}\label{eq:6.2.15} |\mathbf{F}(\mathbf{X})-\mathbf{F}(\mathbf{Y})|\ge (r-\epsilon) |\mathbf{X}-\mathbf{Y}|\mbox{\quad if\quad} \mathbf{X},\mathbf{Y}\in B_\delta(\mathbf{X}_{0}). \end{equation}


If $\mathbf{F}:\R^n\to\R^m$ is continuously differentiable on an open set containing a compact set $D,$ then there is a constant $M$ such that \begin{equation}\label{eq:6.2.18} |\mathbf{F}(\mathbf{Y})-\mathbf{F}(\mathbf{X})|\le M|\mathbf{Y}-\mathbf{X}| \mbox{\quad if\quad}\mathbf{X},\mathbf{Y}\in D. \end{equation}


Suppose that $|f(\mathbf{X})|\le M$ if $\mathbf{X}$ is in the rectangle $$ R=[a_1,b_1]\times [a_2,b_2]\times\cdots\times [a_n,b_n]. $$ Let ${\bf P}=P_1\times P_2\times\cdots\times P_n$ and ${\bf P}'= P_1'\times P_2'\times\cdots\times P_n'$ be partitions of $R,$ where $P_j'$ is obtained by adding $r_j$ partition points to $P_j,$ $1\le j\le n.$ Then \begin{equation}\label{eq:7.1.16} S({\bf P})\ge S({\bf P}')\ge S({\bf P})-2MV(R)\left(\sum_{j=1}^n \frac{r_j}{ b_j-a_j}\right)\|{\bf P}\| \end{equation} and \begin{equation}\label{eq:7.1.17} s({\bf P})\le s({\bf P}')\le s({\bf P})+2MV(R)\left(\sum_{j=1}^n \frac{r_j }{ b_j-a_j}\right)\|{\bf P}\|. \end{equation}


If $f$ is bounded on a rectangle $R$ and $\epsilon>0,$  there is  a $\delta>0$ such that \vspace{4pt} $$ \overline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}\le S({\bf P})<\overline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}+\epsilon $$ \vspace{4pt} and \vspace{4pt} $$ \underline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}\ge s({\bf P})> \underline{\int_R}\, f(\mathbf{X})\,d\mathbf{X}-\epsilon $$ \vspace{4pt} if $\|{\bf P}\|<\delta.$


The union of finitely many sets with zero content has zero content$.$


Suppose that $S$ is contained in a bounded set $T$ and $f$ is integrable on $S.$ Then  $f_S$ $($see $\eqref{eq:7.1.36})$ is integrable on $T,$ and $$ \int_T f_S(\mathbf{X})\,d\mathbf{X}=\int_S f(\mathbf{X})\,d\mathbf{X}. $$


Suppose that  $K$ is a bounded set with zero content and $\epsilon,$ $\rho>0.$  Then there are cubes $C_1,$ $C_2,$ \dots$,$ $C_r$ with edge lengths $<\rho$ such that  $C_j\cap K\ne\emptyset,$ $1\le j\le r,$ \begin{equation}\label{eq:7.3.5} K\subset\bigcup_{j=1}^r C_j, \end{equation} and $$ \sum_{j=1}^r V(C_j)<\epsilon. $$


Suppose that  $\mathbf{G}: \R^n\to \R^n$ is continuously differentiable on a bounded  open set $S,$ and let $K$ be a closed subset of $S$ with zero content$.$ Then $\mathbf{G}(K)$ has zero content.


A nonsingular $n\times n$ matrix $\mathbf{A}$ can be written as \begin{equation}\label{eq:7.3.10} \mathbf{A}=\mathbf{E}_k\mathbf{E}_{k-1}\cdots\mathbf{E}_1, \end{equation} where each $\mathbf{E}_i$ is a matrix that can be obtained from the $n\times n$ identity matrix $\mathbf{I}$ by one of the following operations$:$ \begin{alist} \item % (a) interchanging two rows of $\mathbf{I};$ \item % (b) multiplying a row of $\mathbf{I}$ by a nonzero constant$;$ \item % (c) adding a multiple of one row of $\mathbf{I}$ to another$.$ \end{alist}


Suppose that  $\mathbf{G}:\E^n\to \R^n$ is regular  on a cube $C$ in $\E^n,$ and let $\mathbf{A}$ be a nonsingular $n\times n$ matrix$.$ Then \begin{equation}\label{eq:7.3.29} V(\mathbf{G}(C))\le |\det(\mathbf{A})|\left[\max \set{\|\mathbf{A}^{-1}\mathbf{G}'(\mathbf{Y})\|_\infty}{\mathbf{Y}\in C} \right]^n V(C). \end{equation}


If $\mathbf{G}:\E^n\rightarrow \R^n$  is regular on a cube $C$ in $\R^n,$ then \begin{equation}\label{eq:7.3.32} V(\mathbf{G}(C))\le\int_C |JG(\mathbf{Y})|\,d\mathbf{Y}. \end{equation}


Suppose that  $S$ is Jordan measurable and $\epsilon,$  $\rho>0.$  Then there are cubes $C_1,$ $C_2,$ \dots$,$ $C_r$ in $S$ with edge lengths $<\rho,$ such that  $C_j\subset  S,$ $1\le j\le r,$ $C_i^0\cap C_j^0=\emptyset$ if $i\ne j,$ and \begin{equation} \label{eq:7.3.35} V(S)\le\sum_{j=1}^r V(C_j)+\epsilon. \end{equation}


Suppose that  $\mathbf{G}: \E^n\to \R^n$ is regular on a compact Jordan measurable set $S$ and $f$ is continuous and nonnegative on $\mathbf{G}(S).$ Let \begin{equation}\label{eq:7.3.37} Q(S)=\int_{\mathbf{G}(S)} f(\mathbf{X})\,d\mathbf{X}-\int_S  f(\mathbf{G}(\mathbf{Y})) |J\mathbf{G}(\mathbf{Y})|\,d\mathbf{Y}. \end{equation} Then $Q(S)\le0.$


Under the assumptions of Lemma~$\ref{thmtype:7.3.13},$ $Q(S)\ge0.$


Suppose that  $\mu_1,$ $\mu_2,$ \dots$,$ $\mu_n$ and $\nu_1,$ $\nu_2,$ \dots$,$ $\nu_n$ are nonnegative  numbers$.$ Let $p>1$ and $q=p/(p-1);$ thus$,$ \begin{equation} \label{eq:8.1.5} \frac{1}{p}+\frac{1}{q}=1. \end{equation}  Then \begin{equation} \label{eq:8.1.6} \sum_{i=1}^n \mu_i\nu_i\le\left(\sum_{i=1}^n\mu_i^p\right)^{1/p} \left(\sum_{i=1}^n \nu_i^q\right)^{1/q}. \end{equation}


Suppose that  $u_1,$ $u_2,$ \dots$,$ $u_n$ and $v_1,$ $v_2,$ \dots$,$ $v_n$ are nonnegative  numbers and $p>1.$ Then \begin{equation} \label{eq:8.1.8} \left(\sum_{i=1}^n(u_i+v_i)^p\right)^{1/p} \le\left(\sum_{i=1}^n u_i^p\right)^{1/p} +\left(\sum_{i=1}^n v_i^p\right)^{1/p}. \end{equation}


If $a$ and $b$ are any two real numbers$,$ then \begin{equation} \label{eq:1.1.4} |a-b|\ge\big||a|-|b|\big| \end{equation} and \begin{equation} \label{eq:1.1.5} |a+b|\ge\big||a|-|b|\big|. \end{equation}


If $f$ is continuous on a set $T,$ then  $f$ is  uniformly continuous on any finite closed interval contained in $T.$


If $f'$ is integrable on $[a,b],$ then $$ \int_a^b f'(x)\,dx=f(b)-f(a). $$


If $\sum a_n$ converges$,$ then $\lim_{n\to\infty}a_n=0.$


If $\sum a_n$ converges$,$ then for each $\epsilon>0$ there is an integer $K$ such that $$ \left|\sum_{n=k}^\infty a_n\right|<\epsilon\mbox{\quad if\quad} k\ge K; $$ that is$,$ $$ \lim_{k\to\infty}\sum_{n=k}^\infty a_n=0. $$


Suppose that  $a_n\ge0$ and $b_n>0$ for $n\ge k,$ and $$ \lim_{n\to\infty}\frac{a_n}{ b_n}=L, $$ where $0<L<\infty.$ Then $\sum a_n$ and $\sum b_n$ converge or diverge together$.$


Suppose that  $a_n>0\ (n\ge k)$ and $$ \lim_{n\to\infty}\frac{a_{n+1}}{ a_n}=L. $$ \vskip-1em Then \begin{alist} \item % (a)  $\sum a_n<\infty$ if  $L<1.$ \item % (b)  $\sum a_n=\infty$  if  $L>1.$ \end{alist} The test is inconclusive if $L=1.$


The series $\sum a_nb_n$ converges if $a_{n+1}\le a_n$ for $n\ge k,$ $\lim_{n\to\infty}a_n=0,$ and $$ |b_k+b_{k+1}+\cdots+b_n|\le M,\quad n\ge k, $$ for some constant $M.$


Alternating Series Test

The series $\sum (-1)^na_n$ converges if $0\le a_{n+1}\le a_n$ and $\lim_{n\to\infty} a_n=0.$


If $\{F_n\}$ converges uniformly to $F$ on $S$ and each $F_n$ is continuous on $S,$ then so is $F;$ that is$,$ a uniform limit of continuous functions is continuous.


If $\sum f_n$ converges uniformly on $S,$ then $\lim_{n\to\infty}\|f_n\|_S=0.$


The series $\sum_{n=k}^\infty  f_ng_n$ converges uniformly on $S$ if $$ f_{n+1}(x)\le f_n(x),\quad x\in S,\quad n\ge k, $$ $\{f_n\}$ converges uniformly to zero on $S,$ and $$ \|g_k+g_{k+1}+\cdots+g_n\|_S\le M,\quad n\ge k, $$ for some constant $M.$


If $$ f(x)=\sum^\infty_{n=0} a_n(x-x_0)^n,\quad |x-x_0|<R, $$ then $$ a_n=\frac{f^{(n)}(x_0)}{ n!}. $$


Uniqueness of Power Series

If \begin{equation}\label{eq:4.5.13} \sum^\infty_{n=0} a_n(x-x_0)^n=\sum^\infty_{n=0} b_n(x-x_0)^n \end{equation} for all $x$ in some interval $(x_0-r,x_0+r),$ then \begin{equation}\label{eq:4.5.14} a_n=b_n,\quad n\ge0. \end{equation}


If $\mathbf{X},$ $\mathbf{Y},$ and $\mathbf{Z}$ are  in $\R^n,$ then $$ |\mathbf{X}-\mathbf{Z}|\le |\mathbf{X}-\mathbf{Y}|+|\mathbf{Y}-\mathbf{Z}|. $$


If $\mathbf{X}$ and $\mathbf{Y}$ are  in $\R^n,$ then $$ |\mathbf{X}-\mathbf{Y}|\ge\left| |\mathbf{X}|-|\mathbf{Y}|\right|. $$


Under the assumptions of Theorem~$\ref{thmtype:5.4.3},$ \begin{equation} \label{eq:5.4.8} \frac{\partial h(\mathbf{U}_0)}{\partial u_i}=\sum_{j=1}^n \frac{\partial f(\mathbf{X}_0) }{\partial x_j} \frac{\partial g_j(\mathbf{U}_0)}{\partial u_i},\quad 1\le i \le m. \end{equation}


If $f_{x_1},$ $f_{x_2},$ \dots$,$ $f_{x_n}$ are identically zero in an open region $S$ of $\R^n,$ then $f$ is constant in $S.$


Suppose that  $f,$ $f_x,$ and $f_y$ are differentiable in a neigborhood of a critical point $\mathbf{X}_0=(x_0,y_0)$ of $f$ and $f_{xx},$ $f_{yy},$ and $f_{xy}$ are continuous at $(x_0,y_0).$ Let $$ D=f_{xx}(x_0,y_0)f_{xy}(x_0,y_0)-f^2_{xy}(x_0,y_0). $$ Then \begin{alist} \item % (a)  $(x_0,y_0)$ is a local extreme point of $f$ if $D>0;$ $(x_0,y_0)$ is a local minimum point if $f_{xx}(x_0,y_0)>0$, or a local maximum point if $f_{xx}(x_0,y_0)<0.$ \item % (b) $(x_0,y_0)$ is not a local extreme point of $f$ if $D<0.$ \end{alist}


If $\mathbf{F}$ is continuously differentiable on a neighborhood  of $\mathbf{X}_0$ and $J\mathbf{F}(\mathbf{X}_0)\ne 0,$ then there is an open neighborhood $N$ of $\mathbf{X}_0$ on which the conclusions of Theorem~$\ref{thmtype:6.3.4}$ hold$.$


Suppose that  $f:\R^{n+1}\to \R$ is continuously differentiable on an open set containing $(\mathbf{X}_0,u_0),$ with $f(\mathbf{X}_0,u_0)=0$ and $f_u(\mathbf{X}_0,u_0)\ne0$. Then there is a neighborhood $M$ of $(\mathbf{X}_0,u_0),$ contained in $S,$ and a neighborhood $N$ of $\mathbf{X}_0$ in $\R^n$ on which is defined a unique continuously differentiable function $u=u(\mathbf{X}):\R^n\to \R$ such that $$ (\mathbf{X},u(\mathbf{X}))\in M\mbox{\quad and \quad}  f_u(\mathbf{X},u(\mathbf{X}))\ne0,\quad\mathbf{X}\in N, $$ $$ u(\mathbf{X}_0)=u_0, \mbox{\quad and \quad} f(\mathbf{X},u(\mathbf{X}))=0,\quad\mathbf{X}\in N. $$ The partial derivatives of $u$ are given by $$ u_{x_i}(\mathbf{X})=-\frac{f_{x_i}(\mathbf{X},u(\mathbf{X}))}{ f_u(\mathbf{X},u(\mathbf{X}))},\quad 1\le i\le n. $$


Suppose that  $f$ is integrable on sets $S_1$ and $S_2$ such that $S_1\cap S_2$ has zero content$.$ Then $f$ is integrable on $S_1\cup S_2,$ and $$ \int_{S_1\cup S_2} f(\mathbf{X})\,d\mathbf{X}= \int_{S_1} f(\mathbf{X})\,d\mathbf{X}+ \int_{S_2} f(\mathbf{X})\,d\mathbf{X}. $$


If $f$ is integrable on $[a,b] \times [c,d],$ then $$ \int_a^b dx\int_c^d f(x,y)\,dy=\int_c^d dy\int_a^b f(x,y)\,dx, $$ provided that $\int_c^d f(x,y)\,dy$ exists for $a\le x\le b$ and $\int_a^b f(x,y)\,dx$ exists for $c\le y\le d.$ In particular$,$ these hypotheses hold if $f$ is continuous on $[a,b]\times [c,d].$


If $f$ is bounded and continuous on a bounded Jordan measurable set $S,$ then $f$ is integrable on $S.$


A set $D$  is  {\it dense in the reals\/} if every open interval $(a,b)$  contains a member of $D$.


Let $S$ and $T$ be sets. \begin{alist} \item % (a) $S$ {\it contains\/} $T$, and we write $S\supset T$ or $T\subset S$, if every member of $T$ is also in $S$. In this case, $T$ is a {\it subset\/} of $S$. \item % (b)  $S-T$ is the set of elements that are in $S$ but not in $T$. \item % (c) $S$ {\it equals\/} $T$, and we write $S=T$, if $S$ contains $T$ and $T$ contains $S$; thus, $S=T$ if and only if $S$ and $T$ have the same members. \newpage \item % (d)  $S$ {\it strictly contains\/} $T$ if $S$ contains $T$ but $T$ does not contain $S$; that is, if every member of  $T$ is also in $S$, but at least one member of $S$ is not in $T$ (Figure~\ref{figure:1.3.1}). \item % (e) The {\it complement\/}  of $S$, denoted by $S^c$, is the set of elements in the universal set that are not in $S$. \item % (f)  The {\it union\/} of $S$ and $T$, denoted by $S\cup T$, is the set of elements in at least one of $S$ and $T$ (Figure~\ref{figure:1.3.1}\part{b}). \item % (g) The {\it intersection\/}  of $S$ and $T$, denoted by $S\,\cap\, T$, is the set of elements in both $S$ and $T$ (Figure~\ref{figure:1.3.1}\part{c}). If $S\cap T=\emptyset$ (the empty set), then $S$ and $T$ are  {\it disjoint sets\/} (Figure~\ref{figure:1.3.1}\part{d}). \item % (h)  A set with only one member $x_0$ is a {\it singleton set\/},  denoted by $\{x_0\}$. \end{alist}


If $x_0$ is a real number and $\epsilon>0$, then the open interval $(x_0-\epsilon, x_0+\epsilon)$ is an {\it $\epsilon$-neighborhood\/} of $x_0$. If a set $S$ contains an $\epsilon$-neighborhood of $x_0$, then $S$ is a {\it neighborhood\/} of $x_0$, and $x_0$ is an {\it interior point\/} of $S$ (Figure~\ref{figure:1.3.2}). The set of interior points of $S$ is the {\it interior\/} of $S$, denoted by $S^0$. If every point of $S$ is an interior point (that is, $S^0=S$), then $S$ is {\it open\/}.  A set $S$ is \emph{closed} if $S^c$ is open.


R}$. Then \begin{alist} \item % (a)  $x_0$ is a {\it limit point\/} of $S$ if every deleted neighborhood of $x_0$ contains a point of~$S$. \item % (b) $x_0$ is a {\it boundary point\/} of $S$ if every neighborhood of $x_0$ contains at least one point in $S$ and one not in $S$. The set of boundary points of $S$ is the {\it boundary\/} of $S$, denoted by $\partial S$. The {\it closure\/} of $S$, denoted by $\overline{S}$, is $\overline{S}=S\cup \partial S$. \item % (c) $x_0$ is an \emph{isolated point} of $S$ if $x_0\in S$  and there is a neighborhood of $x_0$ that contains no other point of $S$. \item % (d) $x_0$ is \emph{exterior} to $S$ if $x_0$ is in the interior of $S^c$. The collection of such points is the {\it exterior\/} of $S$. \end{alist}


If  $D_f\cap D_g\ne \emptyset,$  then $f+g,$ $f-g,$ and $fg$ are defined on $D_f\cap D_g$ by \begin{eqnarray*} (f+g)(x)\ar= f(x)+g(x),\\ (f-g)(x)\ar= f(x)-g(x),\\ \noalign{\hbox{and}} (fg)(x)\ar= f(x)g(x). \end{eqnarray*} The quotient $f/g$ is defined by $$ \left(\frac{f}{ g}\right) (x)=\frac{f(x)}{ g(x)} $$ for $x$ in $D_f\cap D_g$ such that $g(x)\ne0.$


We say that $f(x)$ {\it approaches the limit $L$ as $x$ approaches\/} $x_0$, and write $$ \lim_{x\to x_0} f(x)=L, $$ if $f$ is defined on some deleted neighborhood of $x_0$ and, for every $\epsilon>0$, there is a $\delta>0$ such that \begin{equation}\label{eq:2.1.4} |f(x)-L|<\epsilon \end{equation} if \begin{equation}\label{eq:2.1.5} 0<|x-x_0|<\delta. \end{equation} Figure~\ref{figure:2.1.1}   depicts the graph of a function for which $\lim_{x \to x_0}f(x)$ exists.


\begin{alist} \item % (a) We say that $f(x)$ {\it approaches the left-hand limit $L$ as $x$ approaches $x_0$ from the left\/}, and write $$ \lim_{x\to x_0-} f(x)=L, $$ if $f$ is defined on some open interval $(a,x_0)$ and, for each $\epsilon>0$, there is a $\delta>0$ such that $$ |f(x)-L|<\epsilon\mbox{\quad if \quad} x_0-\delta<x<x_0. $$ \newpage \item % (b) We say that $f(x)$ {\it approaches the right-hand limit $L$ as $x$ approaches $x_0$ from the right\/}, and write $$ \lim_{x\to x_0+} f(x)=L, $$ if $f$ is defined on some open interval $(x_0,b)$ and, for each $\epsilon>0$, there is a $\delta>0$ such that $$ |f(x)-L|<\epsilon\mbox{\quad if \quad} x_0<x<x_0+\delta. \eqno{\bbox} $$ \end{alist}


We say that $f(x)$ {\it approaches the limit $L$ as $x$ approaches\/} $\infty$, and write $$ \lim_{x\to\infty} f(x)=L, $$ if $f$ is defined on an interval $(a,\infty)$ and, for each $\epsilon>0$, there is a number $\beta$ such that $$ |f(x)-L|<\epsilon\quad\mbox{\quad if \quad} x>\beta. \eqno{\bbox} $$


We say that $f(x)$ {\it approaches $\infty$ as $x$ approaches $x_0$ from the left\/}, and write $$ \lim_{x\to x_0-} f(x)=\infty\mbox{\quad or \quad} f(x_0-)=\infty, $$ if $f$ is defined on an interval $(a,x_0)$ and, for each real number $M$, there is a $\delta>0$ such that $$ f(x)>M\mbox{\quad if \quad} x_0-\delta<x<x_0. $$


Suppose that  $f$  is bounded on $[a,x_0)$, where $x_0$ may be finite or $\infty$. For $a\le x<x_0$, define \begin{eqnarray*} S_f(x;x_0)\ar= \sup_{x\le t<x_0}f(t)\\\arraytext{and}\\ I_f(x;x_0)\ar= \inf_{x\le t<x_0}f(t). \end{eqnarray*} Then the  {\it left limit superior of $f$ at $x_0$} is defined to be $$ \limsup_{x\to x_0-} f(x)=\lim_{x\to x_0-}S_f(x;x_0), $$ and the  {\it left limit inferior of $f$ at $x_0$} is defined to be $$ \liminf_{x\to x_0-}f(x)=\lim_{x\to x_0-}I_f(x;x_0). $$ (If $x_0=\infty$,  we define $x_0-=\infty$.)


\vspace{6pt} \begin{alist} \item % (a) We say that $f$ is {\it continuous at $x_0$\/} if $f$ is defined on an open interval $(a,b)$ containing $x_0$ and $\lim_{x\to x_0}f(x)=f(x_0)$. \item % (b) We say that $f$ is {\it continuous from the left at $x_0$\/} if $f$ is defined on an open interval $(a,x_0)$ and $f(x_0-)=f(x_0)$. \item % (c) We say that $f$ is {\it continuous from the right at $x_0$\/} if $f$ is defined on an open interval $(x_0,b)$ and $f(x_0+)=f(x_0)$. \bbox \end{alist}


A function $f$ is {\it continuous on an open interval\/} $(a,b)$ if it is continuous at every point in $(a,b)$. If, in addition, \begin{eqnarray} f(b-)=f(b)\label{eq:2.2.2}\\ \arraytext{or}\nonumber\\ f(a+)=f(a)\label{eq:2.2.3} \end{eqnarray} \newpage \noindent  then $f$ is {\it continuous on $(a,b]$ or $[a,b)$\/}, respectively. If $f$ is continuous on $(a,b)$ and \eqref{eq:2.2.2} and \eqref{eq:2.2.3} both hold, then {\it $f$ is continuous on $[a,b]$\/}. More generally, if $S$ is a subset of $D_f$ consisting of finitely or infinitely many disjoint intervals, then $f$ is {\it continuous on $S$\/} if $f$ is continuous on every interval in $S$. (Henceforth, in connection with functions of one variable, whenever we say ``$f$ is continuous on $S$'' we mean that $S$ is a set of this kind.)


A function $f$ is {\it piecewise continuous\/} on $[a,b]$ if \begin{alist} \item % (a)  $f(x_0+)$ exists for all $x_0$ in $[a,b)$; \item % (b)  $f(x_0-)$ exists for all $x_0$ in $(a,b]$; \item % (c)  $f(x_0+)=f(x_0-)=f(x_0)$ for all but finitely many points $x_0$ in $(a,b)$. \end{alist} If \part{c} fails to hold at some $x_0$ in $(a,b)$, $f$ has a {\it jump discontinuity at $x_0$\/}. Also, $f$ has a {\it jump discontinuity at $a$\/} if $f(a+)\ne f(a)$ or {\it at\/} $b$ if $f(b-)\ne f(b)$.


Suppose that $f$ and $g$ are functions with domains $D_f$ and $D_g$. If $D_g$ has a nonempty subset $T$ such that  $g(x)\in D_f$ whenever $x\in T$, then the {\it composite function\/} $f\circ g$ is defined on $T$ by $$ (f\circ g)(x)=f(g(x)). $$


A function $f$ is  {\it uniformly continuous\/} on a subset $S$ of its domain if, for every $\epsilon >0$, there is a $\delta>0$ such that $$ |f(x)-f(x')|<\epsilon\mbox{\ whenever }\ |x-x'|<\delta \mbox{\ and }\ x,x'\in S. \eqno{\bbox} $$


A function $f$ is {\it differentiable\/} at an interior point $x_0$ of its domain if the difference quotient $$ \frac{f(x)-f(x_0)}{ x-x_0},\quad x\ne x_0, $$ approaches a limit as $x$ approaches $x_0$, in which case the limit is called the {\it derivative of $f$ at $x_0$\/}, and is denoted by $f'(x_0)$; thus, \begin{equation}\label{eq:2.3.1} f'(x_0)=\lim_{x\to x_0}\frac{f(x)-f(x_0)}{ x-x_0}. \end{equation} It is sometimes convenient to let $x=x_0+h$ and  write \eqref{eq:2.3.1} as $$ f'(x_0)=\lim_{h\to 0}\frac{f(x_0+h)-f(x_0)}{ h}. \eqno{\bbox} $$


\begin{alist} \item % (a) We say that $f$ is {\it differentiable on the closed interval\/} $[a,b]$ if $f$ is differentiable on the open interval $(a,b)$ and $f_+'(a)$ and $f_-'(b)$  both exist. \item % (b) We say that $f$ is {\it continuously  differentiable on\/} $[a,b]$ if $f$ is differentiable on  $[a,b]$, $f'$ is continuous on $(a,b)$, $f_+'(a)=f'(a+)$, and $f_-'(b)=f'(b-)$. \end{alist}


Let $f$  be defined on $[a,b]$. We say that $f$ is {\it Riemann integrable  on\/} $[a,b]$ if there is a number $L$ with the following property: For every $\epsilon>0$, there is a $\delta>0$ such that $$ \left|\sigma-L \right|<\epsilon $$ if $\sigma$ is any Riemann sum of $f$ over a partition $P$ of $[a,b]$ such that $\|P\|<\delta$. In this case, we say that $L$ is {\it the Riemann integral of $f$ over\/} $[a,b]$, and write $$ \int_a^b f(x)\,dx=L. $$


If $f$ is bounded on $[a,b]$ and $P=\{x_0,x_1, \dots,x_n\}$ is a partition of $[a,b]$, let \begin{eqnarray*} M_j\ar=\sup_{x_{j-1}\le x\le x_j}f(x)\\ \arraytext{and}\\ m_j\ar=\inf_{x_{j-1}\le x\le x_j}f(x). \end{eqnarray*} The {\it upper sum of $f$ over $P$\/}  is $$ S(P)=\sum_{j=1}^n M_j(x_j-x_{j-1}), $$ and the {\it upper integral of $f$ over\/}, $[a,b]$, denoted by $$ \overline{\int_a^b} f(x)\,dx, $$ is  the infimum of all upper sums.  The {\it lower sum of $f$ over $P$\/} is $$ s(P)=\sum_{j=1}^n m_j(x_j-x_{j-1}), $$ and the {\it lower integral of $f$ over\/} $[a,b]$, denoted by $$ \underline{\int_a^b}f(x)\,dx, $$ is the supremum of all lower sums. \bbox


Let $f$ and $g$ be defined on $[a,b]$. We say that $f$ is {\it Riemann}--\href{http://www-history.mcs.st-and.ac.uk/Mathematicians/Stieltjes.html} {\it Stieltjes} {\it integrable with respect to $g$ on\/} $[a,b]$ if there is a number $L$ with the following property: For every $\epsilon>0$, there is a $\delta>0$ such that \begin{equation} \label{eq:3.1.15} \left|\sum_{j=1}^n f(c_j)\left[g(x_j)-g(x_{j-1})\right]-L \right|< \epsilon, \end{equation} provided only that $P=\{x_0,x_1, \dots,x_n\}$ is a partition of $[a,b]$ such that $\|P\|<\delta$ and $$ x_{j-1}\le c_j\le x_j,\quad j=1,2, \dots,n. $$ In this case, we say that $L$ is {\it the Riemann--Stieltjes integral of $f$ with respect to $g$ over\/} $[a,b]$, and write $$ \int_a^b f(x)\,dg (x)=L. $$ The sum $$ \sum_{j=1}^n f(c_j)\left[g(x_j)-g(x_{j-1})\right] $$ in \eqref{eq:3.1.15} is  {\it a Riemann--Stieltjes sum of $f$ with respect to $g$ over the partition~$P$\/}.


If $f$ is locally integrable on $[a,b)$, we define \begin{equation}\label{eq:3.4.1} \int_a^b f(x)\,dx= \lim_{c\to b-}\int_a^c f(x)\,dx \end{equation} if the limit exists (finite). To include the case where $b=\infty$, we adopt the convention that $\infty-=\infty$. \bbox


If $f$ is locally integrable on $(a,b]$, we define $$ \int_a^b f(x)\,dx=\lim_{c\to a+}\int_c^b f(x)\,dx $$ provided that the limit exists (finite).  To include the case where $a=-\infty$, we adopt the convention that $-\infty+=-\infty$.


If $f$ is locally integrable on $(a,b),$ we define $$ \int_a^b f(x)\,dx=\int_a^\alpha f(x)\,dx+\int_\alpha^b f(x)\,dx, $$ where $a<\alpha<b$, provided that both improper integrals on the right exist (finite). \bbox


We say that $f$ is {\it absolutely integrable on\/} $[a,b)$ if $f$ is locally integrable on $[a,b)$ and $\int_a^b|f(x)|\,dx<\infty$.  In this case we also say that $\int_a^b f(x)\,dx$ {\it converges absolutely\/} or {\it is absolutely convergent\/}.


If $f$ is bounded on $[a,b]$, the {\it oscillation of $f$ on\/} $[a,b]$ is defined by $$ W_f[a,b]=\sup_{a\le x,x'\le b}|f(x)-f(x')|, $$ which can also be written as $$ W_f[a,b]=\sup_{a\le x\le b}f(x)-\inf_{a\le x\le b}f(x) $$ \newpage \noindent ( Exercise~\ref{exer:3.5.1}). If $a<x<b$,  the {\it oscillation of $f$ at $x$\/} is defined by $$ w_f(x)=\lim_{h\to0+}W_f(x-h,x+h). $$ The corresponding definitions for $x=a$ and $x=b$ are $$ w_f(a)=\lim_{h\to0+}W_f(a,a+h)\mbox{\quad and \quad} w_f(b)=\lim_{h\to0+}W_f(b-h,b). \eqno{\bbox} $$


A subset $S$ of the real line is {\it of Lebesgue measure zero\/} if for every $\epsilon>0$ there is a finite or infinite sequence of open intervals $I_1$, $I_2$, \dots\ such that \begin{equation} \label{eq:3.5.8} S\subset\bigcup_j I_j \end{equation} and \begin{equation} \label{eq:3.5.9} \sum_{j=1}^n L(I_j)<\epsilon,\quad n\ge1. \end{equation}