Logistic Regression

Logistic Regression is the way to get the solution for Classification problem.

Why logistic regression?

If we use linear regression for classification problem which contain only discrete values, it’ll give wavy curve which is not suitable.Hence, we use Logistic Regression for classification.

Cost function will also be changed in logistic regression, it will look like:- J(θ)=1/m∑Cost(hθ(x(i)),y(i))

Cost(hθ(x),y)=−log(hθ(x)) if y=1

Cost(hθ(x),y)=−log(1−hθ(x)) if y=0

With the help of above formulas, we can also write the simplified cost function and gradient descent also.

Cost Function: J(θ)=−1/m​∑m​[y(i)log(hθ​(x(i)))+(1−y(i))log(1−hθ​(x(i)))]

Gradient Descent: Repeat{ \

θj:=θj−α/m∑(hθ(x(i))−y(i))x(i)j

}

“Conjugate gradient”, “BFGS”, and “L-BFGS”are more sophisticated and faster way to get the output but it needs more expertise thengradient descent, you can apply this algorithm only when you are well aware of numerical problems. Until, you can use predefined libraries which are already provided in Octave.

You can design your own function to get the result in an efficient manner.