Monday, October 24, 2011

LINEAR REGRESSION WITH ONE VARIABLE

REVISED: Sunday, March 3, 2013

You will learn Linear Regression with one variable.

I. LINEAR REGRESSION WITH ONE VARIABLE

Linear Regression is Supervised Learning because you are given the "right answer" for each example in the data.

A Regression Problem has Predicted Real-Value Output.

A Classification Problem has Discrete-Valued Output.

A. How To Choose θ_i's

Choose θ₀, θ₁ so that h_θ( x ) is close to the y value of the training examples ( x, y ).

Minimize J (θ₀, θ₁) = (1/(2m)) ∑ _{(i = 1 to m)} h_θ(x⁽ⁱ⁾-y⁽ⁱ⁾) ²
h_θ(x) for fixed θ₁, is a function of x.
J(θ₁) is a function of the parameter θ₁.

B. Examples

m = 3
Hypothesis: h_θ(x) = θ_{0 +}θ₁x
Parameters: θ₀, θ₁
The cost function is
J (θ_0,θ₁) = (1/(2m)) ∑ _{(i = 1 to m)} h_θ(x⁽ⁱ⁾-y⁽ⁱ⁾) ²
Goal is to minimize J (θ_0,θ₁)

You have learned Linear Regression with one variable.

Elcric Otto Circle

--> --> -->

How to Link to My Home Page

It will appear on your website as:
"Link to ELCRIC OTTO CIRCLE's Home Page"

Thursday, October 20, 2011

MATRIX MULTIPLICATION

REVISED: Sunday, March 3, 2013

You will learn Matrix Multiplication.

I. MATRIX VECTOR MULTIPLICATION

A. Generic Example

As shown below the multiplication of a 3x2 Matrix A, by a 2x1 Vector b, results in a 3x1 Vector c.

   A * b =   c
1   2 15
1 2 3 15
4 5 6 42

mxn matrix nx1matrix m-dimensional
(m rows, (n-dimensioal vector
  n columns) vector)

To get c_i, multiply A's i^th row with elements of vector b, and add them up.

The multiplication steps to create vector c are shown below:
  c
(1*3) + (2*6) = 3 + 12 = 15
(1*3) + (2*6) = 3 + 12 = 15
(4*3) + (5*6) = 12 + 30 = 42

For this to work, the number of columns in Matrix A, has to match the number of rows in Matrix b.

B. Prediction Example

Our hypothesis is that we can predict the sales price of houses based on their square footage as follows:

h_θ(x) = -20 + 0.35x

House sizes:
5678
9123
4567

S is our 3x2 house square footage matrix.
h is our 2x1 hypothesis matrix.
p is our 3x1 matrix product prediction of the sales price of houses resulting from the matrix multiplication of S*h.

S h p
1 5678 1967.30
1 9123 -20 3173.00
1 4567 0.35 1578.40

(-20 * 1) + (0.35 * 5678) = -20 + 1987.30 = 1967.30
(-20 * 1) + (0.35 * 9123) = -20 + 3193.00 = 3173.00
(-20 * 1) + (0.35 * 4567) = -20 + 1598.40 = 1578.40

As shown below we get the same answer using linear algebra.

h_θ(5678) = -20 + (0.35 * 5678) = -20 + 1987.30 = 1967.30
h_θ(9123) = -20 + (0.35 * 9123) = -20 + 3193.00 = 3173.00
h_θ(4567) = -20 + (0.35 * 4567) = -20 + 1598.40 = 1578.40

C. Types

1. Supervised Learning

Supervised Learning is when you are given the right answer for each example in the data set.

2. Classification

Classification is when you have discrete-valued output.

3. Regression Problem

Predict real-valued output.

d. Notation

m = Number of training examples, rows.

(x,y) = One training example, one row.

(x⁽ⁱ⁾, y⁽ⁱ⁾) = i^th training example. The superscript refers to the index of the training set, the i^th row.

x's = "input" variable/features.

y's = "output" variable / "target" variable.

h = Hypothesis; h is a function that maps from x's to y's.

The hypothesis is used to make predictions.

h_θ (x) = θ_o + θ₁x
shorthand h(x)

θ_o and θ₁are parameters.

h is predicting y is a linear function of x.

Linear regression with one variable, x.

Univariate, one variable, linear regression.

You have learned Matrix Multiplication.

Elcric Otto Circle

--> --> -->

How to Link to My Home Page

It will appear on your website as:
"Link to ELCRIC OTTO CIRCLE's Home Page"

MATRIX SCALAR MATH

REVISED: Sunday, March 3, 2013

You will learn Matrix Scalar Math.

I. MATRIX ADDITION

Shown below are two matrices, A and B.

A B
1 2 1 2
3 4 3 4
5 6 5 6

You can only add two matrices of the same dimension.

The above example is two 3x2 matrices; therefore, we can add them as follows:

(1+1) (2+2)
(3+3) (4+4)
(5+5) (6+6)

Notice when we add them, the result is a new 3x2 matrix as follows:

C
2 4
6 8
10 12

II. MULTIPLYING A MATRIX BY A SCALAR NUMBER

Scalar means real number.

For example lets multiply the following 3x2 matrix C by the scalar, real number, 3.

C
2 4
6 8
10 12

and our 3x2 product is as follows:

C
(3*2) (3*4)
(3*6 ) (3*8)
(3*10) (3*12)

which becomes:

C
6 12
18 24
30 36

Notice the above result is a matrix of the same 3x2 dimensions.

III. DIVIDING A MATRIX BY A SCALAR NUMBER

Lets divide the following 3x2 matrix C by the scalar, real number, 3.

We can achieve the same results of division by 3, by multiplication by the inverse of 3, which is 1/3.

We already know how to multiply a matrix by a scalar so we are all set.

Lets multiply the following 3x2 matrix C by the scalar, real number, 1/3.

C
6 12
18 24
30 36

becomes:
C
(1/3 * 6 ) (1/3 * 12)
(1/3 * 18) (1/3 * 24)
(1/3 * 30) (1/3 * 36)

which is:

C
2 4
6 8
10 12

IV. COMBINATION OF OPERANDS

Shown below are three, 3x1 matrices, or vectors.
We want to multiply the scalar number 2, times vector a.
We want to divide vector c , by the scalar number 3.
Then we want to add vectors (a + b), and subtract vector c.

a b c
1 2 3
2 * 4 + 5 - 6 / 3
7 8 9

becomes

a b c
(2 * 1) 2 (3*1/3)
(2 * 4) + 5 - (6*1/3)
(2 * 7) 8 (9*1/3)

becomes

a b c
2 2 (3*1/3)
8 + 5 - (6*1/3)
14 8 (9*1/3)

becomes

a b c
2 2 1
8 + 5 - 2
14 8 3

becomes

a b c

(2 + 2) 1
(8 + 5) - 2
(14 + 8) 3

becomes

c
4 1
12 - 2
22 3

becomes

c
(4 - 1)
(12 - 2)
(22 - 3)

becomes vector 3x1 d.

d
3
10
19

You have learned Matrix Scalar Math.

Elcric Otto Circle

-->

How to Link to My Home Page

It will appear on your website as:
"Link to ELCRIC OTTO CIRCLE's Home Page"

MATRIX VERSUS VECTOR

REVISED: Sunday, March 3, 2013

You will learn the difference between a Matrix versus a Vector.

I. MATRIX

A matrix is a rectangular array of numbers written between square brackets [ ].

The dimension of a matrix is written as the number of rows times the number of columns.

A matrix gives you a way to organize, index, and access lots of data.

A matrix is identified rows by columns.

Matrix A_ij is shown below:

C¹ C² C³ C⁴
1 2 3 4 R¹
5 6 7 8 R²
9 1 2 3 R³

The above is a three row four column matrix or 3x4 or
R^3x4.

Matrix elements or entries inside the matrix A_ij are referred to as i^th row j^th column.

Matrix names are upper case.

II. VECTOR

A vector is a special case of a matrix.

A vector is a matrix that only has one column.

Shown below is the three dimensional vector x_i, meaning it has three rows and one column.

C¹
1 R¹
5 R²
9 R³

The above three row one column matrix is a special case of a matrix, called a vector. It is 3x1 or a three dimensional vector R³.

Vector elements or entries inside vector x_i are referred to as i^th row.

There are two generally accepted ways to index a vector; they are called 1-index and 0-index. The difference is the starting number used for the index. If you are using modern day computer programs, most computer programs index starting with zero. If whoever created the vector did not use modern day computer programs they will index starting with 1.

Vector names are lower case.

You have learned the difference between a Matrix versus a Vector.

Elcric Otto Circle

-->

How to Link to My Home Page

It will appear on your website as:
"Link to ELCRIC OTTO CIRCLE's Home Page"