Network
Click a hidden unit to inspect its half-plane−weight+
0output y1
A layer is a matrix. Read it across the rows and you get half-plane classifiers; read it down the columns and you get vectors that add. The mixed picture does both at once — rows then columns.
Select a unit in the diagram to edit its weights by hand.
A network $f:\mathbb{R}^2\to\mathbb{R}$ as a sum of soft linear classifiers
Write the network with hidden width $N$ as $$f(x)=\sigma\!\Big(\textstyle\sum_{j=1}^{N} v_j\,\tanh(w_j\!\cdot\! x+b_j)+c\Big),\qquad x\in\mathbb{R}^2 .$$ Stack the input weights into a matrix $W_1\in\mathbb{R}^{N\times 2}$. Reading $W_1$ by its rows is the geometry this mode shows: the $j$-th row is the vector $w_j$.
Each hidden unit holds an affine form $a_j(x)=w_j\!\cdot\! x+b_j$. Its zero set $\{x:\,a_j(x)=0\}$ is a line with normal $w_j$, and $\tanh$ turns the sign of $a_j$ into a smooth step: the unit reads $+1$ deep on one side, $-1$ on the other, transitioning across the line. So a hidden unit is a soft half-plane indicator, exactly the dashed line and shaded region you see when you select it.
Because $\tanh(w_j\!\cdot\! x+b_j)$ depends on $x$ only through the scalar projection $w_j\!\cdot\! x$, it is constant along every line orthogonal to $w_j$ — a ridge function. The output superposes $N$ such ridges, $$f(x)=\sigma\Big(\textstyle\sum_j v_j\,g_j(w_j\!\cdot\! x)+c\Big),\qquad g_j=\tanh,$$ which is the classical form behind the universal approximation theorem: finite sums of ridge functions are dense in $C(K)$ for compact $K$.
We draw positives from the target density $p$ and negatives uniformly, label them $t\in\{0,1\}$, and minimise the mean squared error $\;\mathcal{L}=\tfrac12\,\mathbb{E}\,(f(x)-t)^2$. The minimiser of this objective is the conditional mean $f^\star(x)=\Pr[\text{positive}\mid x]$, which is monotone in $p$; the level set $\{f=\tfrac12\}$ is the learned boundary of the density's support.
A linear map $y=Wx$ read down the columns of $W$
For a single linear layer $y=Wx$ with $W\in\mathbb{R}^{2\times 3}$, expand the product as $$y \;=\; Wx \;=\; \sum_{i=1}^{3} x_i\,w^{(i)},\qquad w^{(i)}=W e_i,$$ where $w^{(i)}$ is the $i$-th column of $W$ — the image of the basis vector $e_i$. The output is a weighted sum of these columns, drawn head-to-tail in the right panel.
Every attainable $y$ lies in the column space $\operatorname{col}(W)=\operatorname{span}\{w^{(1)},w^{(2)},w^{(3)}\}$. Choosing the inputs $x_i$ just chooses coefficients; vector addition of the scaled columns is the picture of that linear combination.
Two clusters in $\mathbb{R}^3$ are labelled with the targets $e_1,e_2\in\mathbb{R}^2$. Collecting inputs in $X$ (columns are samples) and targets in $T$, we minimise $$\mathcal{L}(W)=\tfrac12\sum_k \lVert Wx_k - t_k\rVert^2 .$$ Setting the gradient $\nabla_W\mathcal{L}=(WX-T)X^\top$ to zero gives the normal equations $W\,XX^\top = T X^\top$, with solution $W = T X^{+}$ via the pseudo-inverse. Classification then uses $\arg\max_k y_k$.
A network $f:\mathbb{R}^2\to\mathbb{R}^2$ that is the row picture composed with the column picture
The two-layer classifier is $$f(x)=U\,\sigma(W_1 x + b)+c,\qquad h(x)=\sigma(W_1x+b)\in(0,1)^N,$$ with $W_1\in\mathbb{R}^{N\times 2}$ (rows $w_j^\top$) and $U\in\mathbb{R}^{2\times N}$ (columns $u^{(j)}$). The same hidden layer is read two ways at once.
Each row $w_j$ with bias $b_j$ defines a half-plane $\{w_j\!\cdot\! x+b_j=0\}$, exactly as in the row picture. The $N$ hyperplanes partition the input plane into cells, and $h_j(x)=\sigma(w_j\!\cdot\! x+b_j)$ is a soft indicator of which side $x$ falls on. This is the middle panel.
The output is a combination of the columns $u^{(j)}$ with input-dependent coefficients, $$y \;=\; \sum_{j=1}^{N} h_j(x)\,u^{(j)} + c,$$ the column picture with the constants $x_i$ replaced by the activations $h_j(x)$. The right panel adds these scaled columns head-to-tail; the sum should land on $e_1$ or $e_2$.
With targets $t\in\{e_1,e_2\}$ we minimise $\mathcal{L}=\tfrac12\lVert f(x)-t\rVert^2$ and decide by $\arg\max_k y_k$. The map $h$ is a learned feature embedding chosen to make the classes linearly separable; $U$ is the linear read-out that sends each class to a basis vector. The decision set $\{y_1=y_2\}$ pulls back through $f$ to a curved boundary in input space — the tinted regions in the middle panel.
As $N$ grows, $\{x\mapsto U\sigma(W_1x+b)+c\}$ is dense in continuous maps on compact sets (Cybenko, Hornik), so the smooth boundary separating the two clusters can take essentially any shape.
Built as a study instrument · all computation runs locally in your browser · Space train/pause · → step