Sparse matrix
http://www.semanlink.net/tag/sparse_matrix
Documents tagged with Sparse matrixIs the future of Neural Networks Sparse? An Introduction
http://www.semanlink.net/doc/2020/02/is_the_future_of_neural_network
> are all the dimensions of my input
vector interacting with all the others? Usually not. So
going sparse maybe useful.
> Convolutional layers are a smart and efficient way to
implement a sparse transformation on an input tensor... Some
other networks contain much larger matrices that may
benefit from sparsity: Transformers
But
> It’s hard to
implement general sparse matrice computations on
GPUs in an efficient way... Easier if the
matrices non-zeros grouped in small
fixed-size blocks
2020-02-05T00:33:11ZRadek Osmulski sur Twitter : "You would expect a difference in row access times depending on the type of a sparse matrix, but I didn't realize the difference would be so big!
https://twitter.com/radekosmulski/status/1124766298469277696
2019-05-05T10:32:30Z