Hello I want to create a DQN for a 2 player strategic board game. I read that experience replay is recommended to break correlation. I want to know how I create the experience replay table?

Can I build it using random agent?(Playing random moves). After I have built the table I want to proceed like that: I want to randomly take one state out of experience replay table, take an action according to epsilon greedy policy. Put in the next state into a second neural network and backpropagate the difference Q values(From second neural network and first) in the first neuronal network. After that I randomly choose another state and repeat the step from above. Is this correct? Thank you