Linear

Binary class

Binary classification with two features \(\{ x_1, x_2 \}\) as input and two classes \(\{ 0, 1 \}\) as output.

$$ z = x_1 w_1 + x_2 w_2 + b $$ $$ \hat{y} = \begin{cases} 1 & \text{if} \space z \geq 0 \\ 0 & \text{otherwise} \\ \end{cases} $$

Blue: \(1\)
Red: \(0\)
X-axis: standardized \(x_1\), range: \([-2, 2]\)
Y-axis: standardized \(x_2\), range: \([-2, 2]\)
Logistic regression: view probability:

\(w_1\):
\(w_2\):
\(b\):
Learning rate:
Logistic regression: regularization parameter:
Examples \((x_1, x_2, y)\):
[[1,1,0] [-1,-1,1] ]

Three classes

Classification with two features \(\{ x_1, x_2 \}\) as input and three classes \(\{ 0, 1, 2 \}\) as output.

Using one-versus-rest (OvR).

Green: \(2\)
Blue: \(1\)
Red: \(0\)
X-axis: standardized \(x_1\), range: \([-2, 2]\)
Y-axis: standardized \(x_2\), range: \([-2, 2]\)
Logistic regression: view probability:

Class \(0\) vs. the rest:
- \(w_1\):
- \(w_2\):
- \(b\):
Class \(1\) vs. the rest:
- \(w_1\):
- \(w_2\):
- \(b\):
Class \(2\) vs. the rest:
- \(w_1\):
- \(w_2\):
- \(b\):
Learning rate:
Logistic regression: regularization parameter:
Examples \((x_1, x_2, y)\):
[[1,1,0] [-1,-1,1] [1,-1,2] ]