Found myself thinking that 'regression is a special case of a neural network' today and I'm not sure if that's mathematically true or not but it was a fun moment regardless #scilearn

Tom Carpenter A neural network with a linear activation function is a linear operator so it is in that sense correct.
However, ordinarily when we discuss NNs we mean those with nonlinear activation functions, otherwise we would just call it a linear operator.