Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[D] Question on Quaternion-Recurrent-Neural-Networks

concerning https://arxiv.org/abs/1806.04418

I understand that the forward pass needs to be adapted for QRNN; but what’s unclear to me:

Why do they derive a backprop algorithm especially for quaternions? Shouldn’t the automatic differentiation frameworks like Pytorch or Tensorflow do that automatically, at least in the case when the activation function is as they used it? In the final end, I guess everything is implemented by “real”-valued matrices and if the forward pass is according to quaternionic multiplication, why would an automatically derived gradient not work? So my question would be is there any difference of their backward pass to pytorch derived backward pass, apart from computational speed or memory consumption? If I understand correctly, in the copy-task of the github repo (https://github.com/Orkis-Research/Quaternion-Recurrent-Neural-Networks) even a pytorch backward pass is used by default?

Thanks for any answer or intuition that either confirms what I think or contradicts me.

submitted by /u/ml23453425245
[link] [comments]