[R] Zero-shot Knowledge Transfer via Adversarial Belief Matching
![[R] Zero-shot Knowledge Transfer via Adversarial Belief Matching](https://b.thumbs.redditmedia.com/b3oas6EAwXd5hg1ut8_aGzvxqHjNjYQiZYs7zfhRHjo.jpg "[R] Zero-shot Knowledge Transfer via Adversarial Belief Matching") |
Processing gif w2ldw0o30r031… TLDR: Our task is to compress a large neural network (teacher) into a smaller one (student), but we assume that the data used to train the teacher is not available anymore. Our solution is to generate pseudo points adversarially (yellow markers above) and use those to match the student (right) to the teacher (left). Paper (with PyTorch code): https://arxiv.org/abs/1905.09768 submitted by /u/polo555 |