I am doing a project on automating the task to sort rocks on a conveyor belt. My initial idea was to setup a camera and train a model (using TensorFlow) on what is a small rock, big rock, quartz rock. And so far my tabletop demo is successful. My goal is to then send the coordinates of the rock to pick out to a robotic arm that will pick out the rock at a given coordinate. Lets assume that the rock picking process is trivial.

My problem is when vendors approached us, they convinced my boss that they know better (usual vendor logic) and they say that its better if we train the machine by observing an actual operator do the work. They didnt go into much detail but when i asked them how they would do that they had vague answers that made no sense, such as “its an AI that learns which rocks to pick, by wtaching the operator it will learn”

They showed us demos of games being played like racing cars around artificial tracks, super mario etc. But from my understanding, wouldnt it be better if we could actually train the algorithm to detect what is a quartz rock and how to detect it in a camera frame and then proceed to remove the rock? I think its much harder to train the algorithm with a weird camera angle watching someone do the job and then determining which rocks to pick.

Recently i saw a user post a video on here showing a generation based car racing track. (https://www.youtube.com/watch?v=wL7tSgUpy8w) Wouldnt that be a generative model and is maybe what’s causing confusion to the vendors?

Im wondering what would be your thoughts