General Mathematics

back to home

Contents
1

REAL ANALYSIS

1.1

CALCULUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

LOGIC

PROBABILITY AND STATISTICS

3.1

BASIC DEFINITIONS: MEAN, VARIANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.2

BASIC DEFINITIONS: CENTRAL MOMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.3

BASIC DEFINITIONS: SKEWNESS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.4

BASIC DEFINITIONS: KURTOSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.5

BASIC DEFINITIONS: JOINT PDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.6

COVARIANCE, CORRELATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.7

GAUSSIAN DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.8

LOG-NORMAL DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.9

EXPONENTIAL DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.10 BOLTZMAN DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10

FUNCTIONAL ANALYSIS
4.1

CONVOLUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.2

CROSS-CORRELATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.3

AUTO-CORRELATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.4

COVARIANCE, CORRELATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
12

CURVILINEAR ORTHOGONAL COORDINATES

REAL ANALYSIS

1.1

CALCULUS

MULTIPLE INTEGRALS The tricky part is the limits of the multiple integral, a good-to-remember is that the outer
most integral is from point to point, then it is from a curve to a cruve, then it is from a surface to a surface, etc. In
this way, the result will be a number. The corresponding curve and surfaces can be determined from the graph or
analytically..
Remark: Complex #
1
;
| 1z | = |z|
Dot/cross product btw complex vectors are undefined, while algebraic product and division are;
Defn: Frobenius Companion Matrix
The companion matrix is defined for monic polynomial, or
p(t ) = s n + cn1 s n1 + ... + c2 s 2 + c1 s + c0
as:

c
n1
1

0
0

cn2
0
1
0
0
0

cn3
0
0
1
0
0

... c1
...
0
...
0
...
0
...
0
...
1

c0

0
0

Note: Find roots of polynomial=find eigenvalues of companion

By Cayley-Hamilton thm, roots of the monic polynomial is the eigenvalue of companion matrix; and the characteristic
polynomial of the companion matrix is the monic polynomial
Remark: Proving Cauchy MVT/MVT
For Cauchy MVT: given two continuous differentiable function scale one of them so Rolles thm condition is meet, i.e
we create a function that is equal at end points. For MVT, let the 2nd function be the secant line and were done.
Remark: Taylor Polynomials residual
the error associated with Taylor n t h degree approximation is exactly f (n+1) (c)(x a)n+1 /(n + 1)!;
Pf: Construct shifted Taylor polynomial function F(t) centered not at a but at t , t [a, x], with degree n. Construct
G(t ) = (x t )n the helping function. Now apply Cauchy MVT on F (t ) paired with G(t ). Make sense of the result and
were done.
Remark: some proposition about functions
if f : X Y, then f (X A) = f (X ) f (A) Y f (A), subset becomes equal if f surjective.
But in general, f (A B) f (A) f (B), equal if f is injective
Theorem: Divergence Thereom & corollary
Partial sign moves into integrant:
R
R
Vector:
F~ d ~S = F~ dV
Scalar:

V
R
V

R
~ f dV
f d ~S =
V

Remark: Curvature of a sinusoid

Curvature at peak of Asin(t + ) is always A 2 , and 0 at intersection with neutral line.
2

Remark: standard deviation

when to use n, n 1? For population deviation, ie the whole population is sampled, use n; for sample deviation, ie
sampling from a population, use n 1, since one the deviation is used to determine mean.
Remark: t-test, based on student t distribution
is used to determine if difference in sample mean is due population mean differences or merely random variation.
: Bessel Functions of order
is defined by the canonical solutions to the ODE:
x 2 y 00 + x y 0 + (x 2 2 )y = 0
Bessel functions are also known as cylindrical harmonics since they appear in the solution to the Laplace Equation in
cylindriacal coordinates. The most important ones are bessel functions of integer and half-integer order.
Airy/Stokes Equation:
i s defined by:
y 00 x y = 0
This is the simplest second-order linear differential equation with a turning point (a point where the character of the solutions
changes from oscillatory to exponential). The two linearly independent solutions, known as the Airy and Bairy functions,
are denoted Ai (x), B i (x) and of the form:
2

Z
t
1
cos
Ai (x) =
+ xt d t
0
3
3

Z
t
t
1
exp + x t + sin
+ xt
dt
B i (x) =
0
3
3

Function Inner product

the inner product, or dot product, between two functions, over an interval, is a functional that takes in two functions, and
output the signed area under the product function over the given interval:
( f g )I

Ib
Ia

f ()g ()d

LOGIC

L AW OF EXCLUDED MIDDLE tertium non datur/ principium tertii exclusi. Bertrand Russell asserts a distinction between
the "law of excluded middle" and the "law of noncontradiction". In The Problems of Philosophy, he cites three "Laws of
Thought" as more or less "self-evident" or "a priori" in the sense of Aristotle:
1. Law of identity: "Whatever is, is."
2. Law of noncontradiction: "Nothing can both be and not be."
3. Law of excluded middle: "Everything must either be or not be." Note that in semantics, every declarative sentence is
either true or not true means that Law of bivalence is satisfied.

PROBABILITY AND STATISTICS

3.1

BASIC DEFINITIONS:

MEAN, VARIANCE

MEAN of a random variable is defined as:

E(x) = x =

x fX (x)d x

or in the averaging sense:

=

f (x, y)d A

VARIANCE of a random variable is defined as:

Var(x) = E[(x x )2 ]
= E(x 2 ) 2x
POPULATION VARIANCE
1. SIMPLE VARIANCE is the population variance of the sample
Var(x) =

n
1X

i =1

(xi x )2

2. UNBIASED SAMPLE VARIANCE if the mean of the sample is unknown, then the simple variance underestimates
, correcting this is the BESSEL CORRECTION :
the variance by a factor n1
n
Var(x) =

n
X

n 1

i =1

(xi x )2

3. BIASED SAMPLE VARIANCE To minimize the mean squared error(MSE) between sample variance and population variance, one can use n + 1. Correcting for bias makes the MSE worse.
Var(x) =

n
X

n +1

i =1

(xi x )2

bounds for variance,

1. if x [0, 1], then the unbiased sample variance has:
Var(x) (1 ),

x [0, 1]

3.2

BASIC DEFINITIONS: CENTRAL MOMENTS

A array of moments can be defined a round the mean:

n =

(x )n f (x)d x

The mean must be defined for the central moment to exist. (Eg Cauchy dist has no central moment)
The 0th moment is simply 0 = 1
The 1st moment is zero. 1 = 0
The 2nd moment is the variance describing how spread out the distribution is. 2 = 2 . This is indifferent to whether
its below the mean or above
The 3rd moment is related to how skew the distribution is, since far values on the positive side will produce a positive
3rd moment.
The 4th moment measures how peaked or tailed a distribution is.

3.3

BASIC DEFINITIONS:

SKEWNESS

1 =

3
3

It is the normalized 3rd central moment.

3.4

BASIC DEFINITIONS:

KURTOSIS

KURTOSIS
=

4
4

It is the normalized 4th central moment. The more peaked the higher the kurtosis.
For a univariate normal distribution, the kurtosis is 3
EXCESS KURTOSIS is defined by

i.e. how peaked relative to Gaussian. It is more popularly cited.

Various excess kurtosis for common distributions:

 D: Laplace distribution, also known as the double exponential distribution, red curve (two straight lines in the
log-scale plot), excess kurtosis = 3
S: hyperbolic secant distribution, orange curve, excess kurtosis = 2
L: logistic distribution, green curve, excess kurtosis = 1.2
N: normal distribution, black curve (inverted parabola in the log-scale plot), excess kurtosis = 0
C: raised cosine distribution, cyan curve, excess kurtosis = -0.593762...
W: Wigner semicircle distribution, blue curve, excess kurtosis = -1
U: uniform distribution, magenta curve (shown for clarity as a rectangle in both images), excess kurtosis = -1.2.

3.5

BASIC DEFINITIONS:

JOINT PDF

JOINTLY DISTRIBUTED RANDOM VARIABLES the joint pdf is defined by:

fX Y (x, y) = fX (x|Y = y) fY (y) = fY (y|X = x) fX (x)
the cumulative desnity function is defined by:
FX Y (x, y) = P (x X , y Y )
the mean of one random variable is the inner product of X and the joint pdf.
Z
x =
x fX Y (x, y) d x d y
R2

the expected value functional is defined by the inner product with the joint pdf:
Z
E[ g (x, y)] = g (x, y) fX Y (x, y) d x d y

3.6

COVARIANCE, CORRELATION

COVARIANCE AND CORRELATION of two jointly distributed random variable is defined by:
Cov(x, y) = E[(X x )(Y y )]
= E(X Y ) x y
Cor(x, y) = E[(X x )(Y y )]/ x y
= Cov(x, y)/ x y
where the expected value is integration of the inner product with joint pdf.

3.7

GAUSSIAN DISTRIBUTION

Gaussian distribution or normal distribution is remarkbly usefeul due to CENTRAL LIMIT THEOREM : the distribution of sampled mean of any distribution converges to normal.
pdf of Gaussian or normal distribution n(, 2 ) is given by:
pdf
1
p e
2

3.8

(x)2

2 2

cdf

x
1
p
1
+
erf
2
2

entropy
1
2

ln(2e 2 )

LOG-NORMAL DISTRIBUTION

The log normal distribution X is one whose natural log is normaly distributed ln(X ) N (, 2 )
Likewise, if Z is a normal distribution, then e Z follows log-normal distribution.

pdf
1
p e
x 2

(ln x)2

2 2

,x >0

cdf

ln x
1
p
1
+
erf
2
2

3.9

EXPONENTIAL DISTRIBUTION

3.10

BOLTZMAN DISTRIBUTION

e +

2
2

/2

2
2
e 1 e 2+

entropy
1
2

+ 12 ln(2 2 ) +

4.1

FUNCTIONAL ANALYSIS

CONVOLUTION

Convolution
Convolution between two functions is an operator taking in two functions, the target function f and the reversed shifted
weight function g , and returning the average as a function of the shifting t :
Z
( f g ) (t ) =
f ()g (t ) d

The convolution can be thought as profile of average of f created by shifting the focus of the weight function g . The
reversion is to make the shifting procede in the positive direction.
Note that convolution commutes for f and g : f g = g f . Pf: a change of variable shows it directly.
Eg: the convolution with the dirac delta function
f (x) (x xi ) =

R3

R3

f () [x xi ]d
f () [(x xi ) ]d

= f (x xi )
Is simply the shifting of the function to another point of reference.

4.2

CROSS-CORRELATION

Cross correlation, or sliding inner product, between two functions is the inner product as a function of the shifting:
Z
f ()g (t + )d
R f g (t ) = ( f ? g )(t )

where f is the complex conjugate of f . For real functions, this is inconsequential. The cross-correlation is the average of the
target function as a function of the shfiting of the weight. The cross-correlation between two discrete functions is analogous:

R f g [n] = ( f ? g )[n]

f [m] g (n + m)

m=

4.3

AUTO-CORRELATION

Auto-correlation is simply the cross-correlation of the target function with itself. The weight is the function iteself:
R f f (t ) = ( f ? f )(t )
In an autocorrelation, which is the cross-correlation of a signal with itself, there will always be a peak at a lag of zero unless
the signal is a trivial zero signal.
10

Mean, Variance, Covariance, Correlation, Auto-Covariance, Auto-coorelation

MEAN of a random variable is defined as:
E(x) = x =

x fX (x)d x

or in the averaging sense:

=

f (x, y)d A

VARIANCE of a random variable is defined as:

Var(x) = E[(x x )2 ]
= E(x 2 ) 2x
JOINTLY DISTRIBUTED RANDOM VARIABLES the joint pdf is defined by:
fX Y (x, y) = fX (x|Y = y) fY (y) = fY (y|X = x) fX (x)
the cumulative desnity function is defined by:
FX Y (x, y) = P (x X , y Y )
the mean of one random variable is the inner product of X and the joint pdf.
Z
x =
x fX Y (x, y) d x d y
R2

the expected value functional is defined by the inner product with the joint pdf:
Z
E[ g (x, y)] = g (x, y) fX Y (x, y) d x d y

4.4

COVARIANCE, CORRELATION

COVARIANCE AND CORRELATION of two jointly distributed random variable is defined by:
Cov(x, y) = E[(X x )(Y y )]
= E(X Y ) x y
Cor(x, y) = E[(X x )(Y y )]/ x y
= Cov(x, y)/ x y
where the expected value is integration of the inner product with joint pdf.

11

CURVILINEAR ORTHOGONAL COORDINATES

The term curvilinear comes from the fact that when perturbing only one coordinates, the resultant trajectory of a
particle is a curve rather than a line.
DEFINITION OF BASIS VECTOR The basis vectors are unit vectors such that:
r
xi

= hi ei

where r is the position vector, xi is the ith coordinate. hi is the arclength scaling factor.
Table 1

Cartesian(x, y, z)
Cylindrical(r, , z)
Spherical(r, , )

h1
1
1
1

h2
1
r
r sin

h3
1
1
r

SPATIAL DERIVATIVES IN DIFFERENT COORDINATES The catch here is that now the basis vectors changes with
coordinate as well! For Cartesian coordinates, basis vectors dont change:
Table 2: Cartesian derivatives of basis vector

ex
ey
ez
CYLINDRICAL

/ x
0
0
0

/ y
0
0
0

/ z
0
0
0

Not so for cylindrical, note the coupling between the eG t and e r :

Table 3: Cylindrical derivatives of basis vector

er
e
ez

/ r
0
0
0

/
e
er
0

/ z
0
0
0

Key observation: when determining r , observe that perturbing by is rotating the unit vector e r by , the
difference e r must have magnitude 1 and in the direction of the tangent to the pertubation curve. Hence we
have the above relation.
SPHERICAL Similar scenario for spherical note that we denote the AZIMUTHAL ANGLE as and the POLE ANGLE
with z axis as :
Table 4: Spherical derivatives of basis vector

er
e
e

/ r
0
0
0

/
sin e
sin e r cos e
cos e

/
e
0
e r

The modifications unique to the spherical coordinates are that when we perturb , we are not rotating e r , e , but their
radial component on the cut plane(constant z), this component has magnitude sin for e r and magnitude cos for
12

e along that radial. Hence when we perturb , we are rotating these less-than-unit vectors, hence the magitude of
the difference e r , will be sin in the direction of e and the magitude of the difference e , will be cos in the
direction of e . When perturbing , however, things are again similar to perturbing the polar angle in cylindrical case.

13