Trivially a node that has a zero weight can be removed as can any links to / from that node.

It should also be possible to eliminate nodes that have a full-value (aka '1' on the 0-1 scale)

I have also seen work where the matrix multiplies during training can have columns "collapsed".

The ultimate question might be applying an "algebraic simplification" of the final network to simplify a post-trained network used for inference.

The idea is to take a path through a network, constructing the equation for that path, reducing it to a shorter path by conbining nodes and weights.

It is certain that a node participates in several (hundred?) paths. In this case it might be useful to "copy" the node so it can be part of a path reduction without affecting other paths.

I believe that in theory some neural networks can be reduced to a single hidden layer[1]. The game would be to algebraically reduce network depth.

[1] Lee, et al. "On the ability of neural nets to express distributions" https://arxiv.org/pdf/1702.07028.pdf (2021)