Math Background

Topics

• We will frequently use the math on this page
• You will want to be familiar with it


• Topics include
• Exponentiation
• Logarithms
• [Mostly SKIP] Combinatorics: permutations, combinations, number of subsets
• Summations
• Floor and ceiling
• Functions
• Miscellaneous:
• Notation and convention
• Positive and non-negative
• Theorems and Lemmas (iff)
• [SKIP] Inductive proofs

EXPONENTIATION

Exponentiation: Definition

• Addition, multiplication, exponentiation:
• Multiplication is repeated addition
• Exponentiation is repeated multiplication


• Example:
• 2 * 3 = 2 + 2 + 2


• 2 ³ = 2 * 2 * 2


• Questions???

Up Arrow: A Digression

• Addition: $a + n = a + \underbrace{1 + \dots + 1}_{n \textrm{ times}} = \underbrace{succ(succ(\dots succ(a)))}_{n \textrm{ times}}$


• Multiplication: $a \times n = \underbrace{a + a + \dots + a}_{n \textrm{ times}}$


• Exponentiation: $a^n = \underbrace{a \times a \times \dots \times a}_{n \textrm{ times}}$
• Tetration $^n a = a^{a^{\dots^a}}$
• Tetration: $^3 2 = 2^{2^2}$


• Knuth Up Arrow:
• $2\uparrow 2 = 2 \times 2 = 2^2 = 4$
• $2\uparrow 3 = 2 \times 2 \times 2 = 2^3$ = 8
• $2\uparrow 4 = 2 \times 2 \times 2 \times 2 = 2^4 = 16$


• $2\uparrow \uparrow 2 = 2 \uparrow 2 = 2^2 = 4$
• $2\uparrow \uparrow 3 = 2 \uparrow 2 \uparrow 2 = 2^{2^2} = 2 \uparrow (2 \uparrow 2) = 2 \uparrow 4 = 16$
• $2\uparrow \uparrow 4 = 2 \uparrow 2 \uparrow 2 \uparrow 2 = 2^{2^{2^2}} = 2 \uparrow (2 \uparrow (2 \uparrow 2)) = 2 \uparrow (2 \uparrow 4) = 2 \uparrow 16 = 65,536$


• $2\uparrow \uparrow \uparrow 2 = 2\uparrow \uparrow 2 = 2 \uparrow 2 = 4$
• $2\uparrow \uparrow \uparrow 3 = 2\uparrow \uparrow (2 \uparrow\uparrow 2) = 2 \uparrow \uparrow 4 = 65,536$
• $2\uparrow \uparrow \uparrow 4 = 2\uparrow\uparrow (2 \uparrow\uparrow (2 \uparrow\uparrow2)) = 2\uparrow\uparrow (2 \uparrow\uparrow 4) = 2\uparrow\uparrow 65,536 = 2^{2^2} 65,536$ times$= ???$


• $2\uparrow \uparrow \uparrow \uparrow 2 = 2\uparrow\uparrow \uparrow 2 = 2\uparrow\uparrow 2 = 2\uparrow 2 = 4$
• $2\uparrow \uparrow \uparrow \uparrow 3 = 2\uparrow\uparrow \uparrow (2 \uparrow\uparrow\uparrow 2) = 2\uparrow\uparrow \uparrow 4 = 2^{2^2} 65,536$ times
• $2\uparrow\uparrow \uparrow \uparrow 4 = 2\uparrow\uparrow \uparrow (2 \uparrow\uparrow\uparrow (2 \uparrow\uparrow\uparrow 2)) = 2\uparrow\uparrow \uparrow (2 \uparrow\uparrow\uparrow 4) = 2\uparrow\uparrow \uparrow ( 2^{2^2} 65,536$ times)


• $2 \uparrow\uparrow\uparrow 3 = 2 \uparrow\uparrow 4 \ $
• $2 \uparrow\uparrow\uparrow\uparrow 3 = 2 \uparrow\uparrow\uparrow 4 \ $
• $2 \uparrow^m n \ = 2 \uparrow^{m-1} (2\uparrow^n(n-1))$

Exponentiation: Properties of Powers

• Properties follow from the definition:
• b^ab^c = b^{a + c}


• b⁰ = 1 [WHY?]


• b^a/b^c = b^{a - c}


• 1 / b^c = b^-c [WHY?]


• (b^a)^c = b^ac = (b^c)^a


• Examples:
• 2² * 2³⁰ = 2³²


• 2¹⁰ / 2⁴ = 2^10-4 = 2⁶ = ?


• 2^-3 = 1 / 2³ = 1 / 8


• (2³)⁴ = 2^3*4 = 2¹² = 2² * 2¹⁰ = 4 * 1024 = 4096
• (2³)⁴ = 8⁴ = 8^{2 * 2} = (8²)² = 64² = 4096

Buzz Numbers

• 2¹⁰ = 1024 ≈ 1000 = 10³


• 2²⁰ = (2¹⁰)² = 1024 ² ≈ 1000 ² = (10³) ²=10⁶ = 1,000,000


• 2³⁰ = ... ≈ ...


• 2³² = ... ≈ ...

LOGORITHMS

Logarithm: Definition

• $log_b x$ is defined as the power to which the base, b, must be raised to produce x


• So, $b^{\log_b x}=x$


• Assume b > 0
• Assume x > 0

Examples

• log₂ 8 = ?
• log₂ 4 = ?
• log₂ 2 = ?
• log₂ 1 = ?


• log₂ 1024 = ?


• log₁₀ 1000 = ?

Logarithm: More Background

• Napier: 1614
• algorithm = logos + arithmos (Greek)
• Replace multiplication by lookup, addition, lookup
• Tables
• More tables
• Multiplicaion by constant factor is addition by constant

Properties of Logarithms

• Properties follow from the definition:
1. log_b 1 = 0, for any b


2. log_b b = 1, for any b


3. log_b (b ^x) = x


4. b ^{log_b x} = x


5. log_b xy = log_b x + log_b y


6. log_b x^y = y log_b x


7. log_b x/y = log_b x - log_b y

Examples of Properties

• 2 ^{log₂ 7} = 7


• log₂ (4 * 8) = log₂ 4 + log₂ 8 = 2 + 3 = 5
• log₂ (4 * 8) = log₂ 32 = 5


• log₂ (16 / 2) = log₂ 16 - log₂ 2 = 4 - 1 = 3
• log₂ (16 / 2) = log₂ 8 = 3


• log₂ (16 ⁵) = 5 * log₂ 16 = 5 * 4 = 20
• log₂ (16 ⁵) = (log₂ (2⁴)⁵ = lg 2^4*5 = log₂ 2²⁰ = 20

More Properties of Logarithms

• These also follow from the definition, but are not so obvious:


1. log_a x * log_b a = log_b x
• Intuition: Consider log₈ 64
• How to prove?


2. log_a x = log_b x / log_b a   [Why?]
3. log_a x = log_b x * log_a b   [Why?]


4. log_a b = (1 / log_b a)   [Why?]


5. a ^{log_b x} = x ^{log_b a}
• This allows us to change n ^{lg x} to x ^{lg n}
• How can we prove it?


• Examples:
• log ₄ 64 * log₂ 4 = 3 * 2 = 6 = log₂ 64
• log ₈ 64 * log₂ 8 = 2 * 3 = 6 = log₂ 64


• log ₄ 64 = log₂ 64 / log₂ 4 = 6 / 2 = 3
• log ₄ 64 = log₂ 64 / log₂ 8 = 6 / 3 = 2


• log ₄ 32 = lg 32 / lg 4 = 5 / 2 = 2.5
• 4^2.5 = 4^2 * 4^0.5 = 16 * 2


• 8 ^{lg 4} = 4 ^{lg 8} = 4 ³ = 64 = 8²

Notation: Lg, Log, and Ln

• In this course, log₂ is most common [Why?]


• We write lg x to mean log₂ x
• lg 4 = 2
• lg 8 = 3
• 2 ^{lg 7} = 7
• lg (4 * 8) = lg 4 + lg 8 = 2 + 3 = 5


• Other notations:
• log x means log₁₀ x
• Common log
• (Or, we don't care about the base)


• ln x means log_e x
• Natural log
• e ≈ 2.71828

Logarithm Intuition

• Definition: $\log_b x$ is the power to which we raise b to get $x$


• Intuition: $\log_b x$ is the number of times $x$ can be divided by $b$ before reaching 1


• Examples:
• log₁₀ 1000 = 3


• log₂ 1024 = 10


• log₂ 4 = 2
• log₂ 8 = 3

Properties of Logarithms Revisited

• Re-examine properties in light of intuition
1. log_b 1 = 0, for any b
2. log_b b = 1, for any b
3. log_b (b ^x) = x
4. b ^{log_b x} = x


5. log_b xy = log_b x + log_b y
6. log_b x^y = y log_b x
7. log_b x/y = log_b x - log_b y


8. log_a x * log_b a = log_b x


9. log_a x = log_b x / log_b a   [Why?]
10. log_a x = log_b x * log_a b   [Why?]


11. log_a b = (1 / log_b a)   [Why?]


12. a ^{log_b x} = x ^{log_b a}

Approximations

What if the power is not an integer?
• log₂ 4 = 2.000
• log₂ 7 ≈ 2.807
• log₂ 8 = 3.000
• Intuition: If x is an integer power of b, it's exactly the number of times to divide, otherwise approximately

Logarithms and Negative Powers

• log₂ 1/8 = -3


• lg (1 / 8) = lg 1 - lg 8 = 0 - 3 = -3


• Intuition makes most sense for x ≥ 1, but easily fits negative powers

Does the Base Matter - All Logs are the Same

• Does it matter what base we use when working with logs?


• Yes, if we need a precise answer:
• lg 1024 = 10
• log 1024 ≈ 3


• No, if we don't care about constants:
• log_a x = log_b x / log_b a
• log_b a is a constant
• Thus, lg n = k log n, where k = 1 / log 2 ≈ 3.33


• Thus: Big O of all algorithms with log performance is the same, regardless of base


• Example: Algorithms that divide input in halves and in thirds both have log behavior and differ only by a constant

More Examples of Powers and Logs

• Properties of powers and logs (repeated from above)
• b^ab^c = b^{a + c}
• b^a/b^c = b^{a - c}
• b^-a = 1 / b^a
• (b^a)^c = b^ac = (b^c)^a


• log_b 1 = 0, for any b
• log_b b = 1, for any b
• log_b (b ^x) = x
• b ^{log_b x} = x


• log_b x^y = y log_b x
• log_b xy = log_b x + log_y b
• log_b x/y = log_b x - log_y b


• a ^{log_b x} = x ^{log_b a}
• log_a x = log_b x / log_b a


• Solve the following:
1. lg(n/4)
2. lg(sqrt(n))
3. lg 2ⁿ
4. 2^{lg n}
5. 2^{2 lg n}
6. lg(4n log n)

Natural Log

• Why use e as a base for natural log?
• Answer: e and ln have many nice properties


• Useful property: ln x = area under curve 1/x between 1 and x
• We can approximate the area under curve from 1 to x with rectangles with base 1


• This allows us to approximate 1 + 1/2 + 1/3 + ... + 1/n, which has no closed form
• 1 + 1/2 + 1/3 + ... + 1/n ≈ ln n


• Useful result (unproved):
• [1/2 + 1/3 + ... + 1/(n-1)] + 1/n < ln n < 1 + [1/2 + 1/3 + ... + 1/(n-1)]


• Bottom Line: sometimes we will use ln n as an approximation for $\displaystyle \sum_{i=1}^n \frac{1}{i}$

COMBINATORICS - MOSTLY SKIP

Permutations

• Definition: ordering of a set of objects


• In practice: how many ways can you order n objects?


• Example - permutations of abc: abc, acb, bac, bca, cab, cba
• How to count?


• Number of permutation of n objects: $P(n) = n!$
• Why?
• Reasoning: n choices for first object, $n-1$ for second, ..., 1 for last

SKIP THE REST

Combinations

• c($n, k$) is the number of ways of choosing k objects from n objects when order does not matter
• The text describes $c(n, k)$ as the number of k-combinations of an n-element set
• Also written as $\binom n k$
• Spoken as "n choose k"


• $\displaystyle c(n, k) = \frac{n!}{(k!(n-k)!)} = \frac{n*(n-1)*(n-2)*...*(n-k+1)} {k!}$


• Reasoning:
• $n$ choices for first object, $n-1$ for second, $n-(k-1)$ for last.
• Divide by $k!$ because order doesn't matter and each group of $k$ elements could have been chosen in $k$ ways


• Binomial Theorem: $\displaystyle (a+b)^n = \sum_{k=0}^{n} c(n, k)a^k b^{n-k}$

Subsets

• Number of subsets of an n-element set: 2ⁿ
• Students from 420: What is the set of these subsets called?

Summations

Important Summation Formula

1. $\displaystyle \sum_{i=l}^{h}1 = \underbrace{1 + 1 + \dots + 1}_{u-l+1 \text{ times}} = h-l+1$
• $\displaystyle \sum_{i=1}^{n}1 = n$
2. $\displaystyle \sum_{i=1}^{n}i = 1 + 2 + \dots + n = \frac{n(n+1)}{2} \approx \frac{n^2}{2}$
3. $\displaystyle \sum_{i=1}^{n}i^2 = 1 + 4 + \dots + n^2 = \frac{n(n+1)(2n+1)}{6} \approx \frac{n^3}{3}$
4. $\displaystyle \sum_{i=1}^{n}i^k = 1 + 2^k + \dots + n^k \approx \frac{n^{k+1}}{k+1}$
5. $\displaystyle \sum_{i=0}^{n}a^i = 1 + a + a^2 + \dots + a^n = \frac{a^{n+1}-1}{a-1}$ (a ≠ 1)
• $\displaystyle \sum_{i=0}^{n}2^k = 2^{n+1}-1$
6. $\displaystyle \sum_{i=1}^{n}i2^i = 1\times 2 + 2\times 2^2 + \dots + n 2^n = (n-1)2^{n+1}+2$
7. $\displaystyle \sum_{i=1}^{n}1/i = 1/1 + 1/2 + \dots + 1/n ≈ \ln n + \gamma$, where $\gamma = 0.5722\dots$ (Euler's constant)
• See above under natural log for intuition on why this is true
8. $\displaystyle \sum_{i=1}^{n}\textrm{lg }i \approx n \lg n$


9. Most important: 1, 2, 5
10. Next most important: 4, 7, 8

Manipulating Summation Formula

1. $\displaystyle \sum_{i=1}^{n}ca_i = c\sum_{i=1}^{n}a_i$
2. $\displaystyle \sum_{i=1}^{n}(a_i \pm b_i) = \sum_{i=1}^{n}a_i \pm \sum_{i=1}^{n}b_i$
3. $\displaystyle \sum_{i=l}^{u}a_i = \sum_{i=l}^{m}a_i + \sum_{i=m+1}^{u}a_i$
4. $\displaystyle \sum_{i=l}^{u}(a_i - a_{i-1}) = a_u - a_{l-1}$

Approximation of a Sum by a Definite Integral

See the text (Appendix A)

Floor and Ceiling Functions

Floor

• Floor($x$) = the largest integer that is ≤ $x$
• floor($x$) is written as $\lfloor x \rfloor$
• On a number line, floor($x$) is the first integer at or left of $x$ (ie towards $-\infty$)
• for positive $x$, floor($x$) truncates the fractional part of $x$

Ceiling

• Ceiling($x$) = the smallest integer that is ≥ x
• ceiling($x$) is written as ⌈x⌉
• On a number line, floor($x$) is the first integer at or right of $x$ (ie toward $\infty$)
• for negative $x$, floor($x$) truncates the fractional part of $x$

Properties of Floor and Ceiling

1. $x - 1 \lt \lfloor x \rfloor \le x \le \lceil x \rceil \lt x + 1$
2. $\lfloor x + n \rfloor = \lfloor x \rfloor + n$ for real x and integer n
• $\lceil x + n \rceil = \lceil x \rceil + n$ for real x and integer n
3. $\lfloor n/2 \rfloor + \lceil n/2 \rceil = n$
4. $\lceil \lg(n+1) \rceil = \lfloor \lg n \rfloor + 1$


5. Any positive number n can be halved (integer division) ⌊lg n⌋ times before reaching 1
• Any positive number, n, that is a integer power of 2 can be halved exactly lg n times before reaching 1

Miscellaneous

Notation for Powers [NIB]

• We (sometimes) write $x$^$y$ to denote $x^y$ and $x$^{$yz$} to denote $x^{yz}$

Postive and Non-negative

• Positive numbers are greater than 0
• Non-negative numbers are greater than or equal 0

Theorems and Lemmas

• p if and only if q means [(if p then q) and (if q then p)]
• In symbols: p ⇔ q means [(p ⇒ q) ∧ (q ⇒ p)]
• p if and only if q is sometimes written as p iff q


• p ⇒ q is NOT equivalent to !p ⇒ !q
• p ⇒ q is equivalent to !q ⇒ !p
• Example: p(x) = x > 2, q(x) = x^2 > 4

Conventions

• lg has high precedence: lg n+1 means (lg n) + 1, not lg(n+1)


• lg lg n means lg (lg n)


• 2^lg_n = n
• lg 2ⁿ = n

Inductive Proofs (ie Mathematical Induction) - SKIP

Inductive Proofs (ie Mathematical Induction)

• Technique for proving a property is true for an infinite set of integers
• Usually the positive or negative integers


• Based on fundamental this property of integers: if 1 and 2 below are true, then 3 is true:


1. Property $p(i)$ is true for $i=1$ [or some other initial value]


2. For any $n ≥ 1$, if Property $p(i)$ is true of $i=n$ then $p(i)$ is true for $i=n+1$,
• that is, if $p(n)$ is true, then $p(n+1)$ is also true


3. Property $p(i)$ holds for all integers $i ≥ 1$ [or other initial value]

Basis, Induction Hypothesis, Inductive Step

• Inductive proofs use the following terms for their parts:
• Basis: 1. above
• Inductive Hypothesis (IH): in 2. above, assuming that $p(n)$ is true
• Inductive Step (IS): in 2. above, proving that $p(n)$ implies $p(n+1)$

Induction Example

• Prove $1+2+\dots+n = n(n+1) / 2$, for all $n ≥ 1$
• Property $p(i)$ is $1+2+...+i = i(i+1) / 2$
• Prove $p(i)$ is true for all $n ≥ 1$


• Basis - $p(0): 1 = 1(1+1)/2 1(2)/2 = 2/2 = 1$


• IH: Assume $p(n): 1+2+...+n = n(n+1)/2$, for some $n ≥ 1$
• IS: Show that IH implies $p(n+1)$: that is, the IH implies $1+2+...+(n+1) = (n+1)((n+1)+1)/2$ $$\begin{align*} 1+2+...+(n+1) & = 1+2+...+n+(n+1) \\ & = n(n+1)/2 + n+1\textrm{, by IH} \\ & = [n(n+1) + 2(n+1)] / 2 \\ & = [(n+1)(n+2)] / 2 \\ & = [(n+1)((n+1)+1)] / 2 \end{align*}$$
  
• Therefore, by the basis and IS, $p(k)$ is true for all $k ≥ 1$


• Can we prove $p(k)$ for $k ≥ 0$? Change $p(k)$ to $0+1+...+k=k(k+1)/2$

Induction: Final Points

• Sometimes there are other ways to prove
• Example A.4: $\Sigma_{i=0}^{n} r^i = \frac{r^{n+1} - 1} {r-1}$


• Sometimes a slightly different IH is used:
• Assume $p(k)$ holds for ALL $k ≤ n$ (not just for $k=n$)