Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] Activation masking (pruning), then how to calculate pruned weights (zeroed weights due to zero activations)

Calculating pruned weights after pruning them is easy, just nonzero_params/all_params * 100%

However I can’t find papers/method to calculate pruned weights via activation-mask based pruning,

Anyone here know how to calculate pruned weights due to pruned activations (masked activations)? This is the case where no weights are pruned directly, but instead weights are pruned by zeroing them by putting mask on activations before them ( because zero * zero = zero)

Unlike ordinary weight pruning, it involves going through zeros through matmul, conv, with strides, padding, and kernel size, etc.

submitted by /u/tsauri
[link] [comments]