pmixer/zan.initializer

Sample to show how input dim(input_dim for FC layer, for CNN, it's prod(tensor.shape[1:])) reciprocal could be used for initialize NN weights rather than sqrt(input_dim_reciprocal) hinted by Xavier:dragon:

Language: Jupyter Notebook - Size: 67.4 KB - Last synced at: 2 months ago - Pushed at: over 4 years ago - Stars: 0 - Forks: 0