I am using BERT for sentence embedding, and currently my best solution is a highly optimized rainbow dictionary, but it’s not scaling well. Attempts to do matching on sub-strings has proved unsuccessful.

I am about to try and train a bidirectional RNN, but that is a huge direction. I was wondering if anyone had advice to look in.