7-Logistic Regression

The difference between linear regression and logistic regression lies in what happens after the above linear function has been calculated. In linear regression, we’re done and the linear function is the output. In logistic regression, we take the result of the linear function and apply a special non-linear function called the sigmoid function to it.

\(s(z)=1÷(1+exp(−z))\)

The mathematical formula for the sigmoid function (see below) may look a bit intimidating but it serves a simple purpose: it converts any number to value between 0 and 1. For example, 0 is converted to 0.5, 10 is converted to 0.9999546, -10 to 0.0000454. This is useful to interpret an outout as a probability (remember, porbabilities are calculated values betewween 0 and 1) adn use this to predict the probability of something happening.

The sigmoid formula is tehn used get the probability of something happening or not, or a mail to be spam or not. There is a simple generalization of the sigmoid function to multiple class labels, which is called softmax.

import math
import numpy as np

x = np.array([4, 3, 0])
c1 = np.array([-.5, .1, .08])
c2 = np.array([-.2, .2, .31])
c3 = np.array([.5, -.1, 2.53])

def sigmoid(z):
    return( 1 / (1 + math.exp(-z)))

#Same as linear regression
z1 = np.dot(x, c1) # (4 × -0.5) + (3 × 0.1) + (0 × 0.08) = -2.0 + 0.3 + 0 = -1.7
z2 = np.dot(x, c2) #  (4 × -0.2) + (3 × 0.2) + (0 × 0.31) = -0.8 + 0.6 + 0 = -0.2
z3 = np.dot(x, c3) #  (4 × 0.5) + (3 × -0.1) + (0 × 2.53) = 2.0 - 0.3 + 0 = 1.7

# Apply sigmoid function to each result
print("for c1 -> "+str(sigmoid(z1)))
print("for c2 -> "+str(sigmoid(z2)))
print("for c3 -> "+str(sigmoid(z3)))

for c1 -> 0.1544652650835347
for c2 -> 0.45016600268752216
for c3 -> 0.8455347349164652

On y yes ot no decision making model, c3 coefficients would be the most likely to output a yes value.