Yeah, there is a lot of work to be done with this project: enemies, genetic algorithm for reproduction, saving weights, extend world, your idea with food is interesting and maybe to add extra actions to agents.
But because of my study and I'm one who do this, progress is slow.
Yes, it is multi-layer perceptron and outer is Q-function.
Actually, I didn't solve local minimum problem yet, but some result I've got when I tune the alpha from 0.1 to 0.0001, but it still there is. So this is one more think to work on it in the future