Skip to main content

Blog

Learn About Our Meetup

5000+ Members

MEETUPS

LEARN, CONNECT, SHARE

Join our meetup, learn, connect, share, and get to know your Toronto AI community. 

JOB POSTINGS

INDEED POSTINGS

Browse through the latest deep learning, ai, machine learning postings from Indeed for the GTA.

CONTACT

CONNECT WITH US

Are you looking to sponsor space, be a speaker, or volunteer, feel free to give us a shout.

[P] Questions about Fully Connected Layers and Dropout

Disclaimer: I’m fairly new to deep learning

I’m trying to implement a model that I read about in a paper in Keras. I understand the Conv2D and MaxPool2D layers up until that point but I’m slightly confused about the fully connected layers. The following is my code for the last 4 layers:

model.add(MaxPool2D(pool_size = (3, 3), strides = (2, 2)))

model.add(Dropout(0.1))

model.add(Dense(256)) (First Dense Layer)

model.add(LeakyReLU(alpha = 0.3))

model.add(Dropout(0.1))

model.add(Dense(256)) (Second Dense Layer)

model.add(LeakyReLU(alpha = 0.3))

model.add(Dropout(0.1))

model.add(Dense(5, activation = ‘softmax’)) (Last Dense Layer)

A) Should the dropout come before or after the Dense/Fully Connected Layers (I’m pretty sure they’re the same thing – correct me if I’m wrong)? The paper says they used dropout in the fully connected layers but it doesn’t mention whether it came before or after.

B) Is my code for the Dense layers correct? The MaxPool layer at the top of my code above takes a 16x16x128 feature map and reduces it to 7x7x128, resulting in an input size of 6272 to the first Dense/FC layer. The first Dense/FC layer should have an output of 256 FC units and the second Dense/FC layer should take this input and output 256 FC units. The last dense layer should take the input of 256 FC units from the second dense layer and output 5 probabilities using a softmax function.

If anything is unclear, please tell me and I will try to clarify. Responses are much appreciated.

submitted by /u/FrontJournalist
[link] [comments]