Setting: I have a convolution of Image I and kernel w, J=conv2D(I,w). I is an image I of size nxn with c channels, while w is a cxcx3x3 matrix, therefore we have c 3×3 filters that can be applied to the image. We use zero-padding so that J has the same dimensionality of I.

What properties does w have to fulfill for the convolution to be invertible? It is clear that at least one such w exists, because 1×1 convolutions are a subset of 3×3 convolutions and since we have c filters, we can encode the identity, which is invertible. It is not enough for the filters to be independent, because as the input has 9c dimensions, we can still loose information.