I’m making a ConvNet that classifies my data into one of classes [A, B, C, D, “other”], “other” being anything that doesn’t fit into [A, B, C, D].

I’m thinking of mapping the input x to a vector “c” in R^4, and then transforming “c” to R⁵ (using a few tiny FC layers) before applying softmax to get the output.

My reasoning is that I don’t want the network attempting “learn” anything specific about the “other” class, I just want it to be a sort of “nor” operation. The network should only focus on learning the features of [A, B, C, D]. The variance in the “other” class is very high so I want it to generalize well.

Thoughts? Papers? Prior experience?