Two Lotteries: What It Takes to Make Sparsity Fast
Suppose you remove 80% of a neural network’s weights and recover its original accuracy. You might reasonably expect it to run much faster. Then you put it on a GPU, and the improvement is small. Sometimes it runs slower. What it takes to turn a smaller network into a faster one.
Sparsity
Accelerators
Co-design
Sparse attention
Read the post