Blog

Learn About Our Meetup

4500+ Members

[D] Is attention just a type of belief propagation?

Attention is mostly discussed in the context of sequences, but I was thinking about what it meant in the context of message passing. Clearly when a certain word has “attention” it is relaying long-distance information, but is that synonymous with something like belief propagation?

submitted by /u/Toast119
[link] [comments]

Next Meetup

 

Days
:
Hours
:
Minutes
:
Seconds

 

Plug yourself into AI and don't miss a beat