This is one of these fantastic "all that is done with so little code?!" moments.
Amazing that this can be done in 300 lines of perfectly readable JS without any libraries. And the author apparently wrote it in 2 days. Great intro to genetic algorithms and reinforcement learning. Universities should teach like that.
Edit:
Somebody has asked about the big picture how the thing works:
You use a neuronal network to decide at each step to flap the birds wings or not. That is the only output. The input is only the birds height (y-position) and the height of the next hole. (https://en.wikipedia.org/wiki/Feedforward_neural_network)
The question now is how to find the correct weights for the net.
That net is not trained in a traditional supervised way like with gradient descent (which would be more complicated). Instead it uses a genetic algorithms to find new weights for neuronal networks of future generations. That is use some of the best individuals. Some randomly generated ones and some created by breeding from existing individuals. (https://en.wikipedia.org/wiki/Genetic_algorithm)
These are the exactly gems that I am constantly in search of, and thrilled every time I come across one.
This is going to turn out to be a rant, but sometimes it feels like there's a culture of solving problems by a philosophy of "let's throw more money at it, more technology at it, more people, more unnecessary abstraction layers, complexity and bloat - it's bound to be solved eventually". And with enough fire power, it usually is - but at what cost... I'm not sure what the source of the problem is though, and it's not only with code. Many times I struggle to understand certain concepts via professional literature, and when I finally understand them, I'm puzzled as to why they are taught in a way that if I didn't know better, I'd assume a deliberate attempt to confuse the reader instead of making the learning simple, intuitive and fun.
Exactly the same applies to code I've had to refactor and sometimes rewrite over the years. Obviously, some of it has to do with trying to get a product out in crazy schedules which requires making compromises, but I can tell the difference between compromise and "I don't really care" attitude, and I'm talking about the latter. It's a pity though - not only for the one that has to sort out and sometimes clean up the mess, but also for the one that creates it. Things are so much more fun where you care and are passionate about what you do, that I'm sorry for those who are missing out on it.
End of rant. Not sure it belongs here but had to get it out of my system :-)
If that's all it was then I wouldn't bother to bring it up. What I'm saying is that more focus on the beauty of a solution (e.g robustness, scalability, flexibility, simplicity) can be a more financially sound solution in the long run than brute force. Of course, I'm not holding my breath for this to become the norm since using brute force to "get what we want now" is popular because it gets fast results (and this not limited to the software industry) - I'm just suggesting the real costs come later.
As for explaining being a tough skill - what I find in common with the previous point is the lack of stress on keeping things as simple and obvious as possible. If you truly understand something, you should be able to convey that understanding assuming you really want to - it just takes investment, usually by asking yourself why it is obvious to you, then putting yourself in the other party's shoes and then finding the shortest and clearest path to bring them from where they are to where you are. In my experience it's not a tough skill - just requires introspection into your own understanding and not assuming anything about what the other person understands.
I was not trying to dismiss what you said. If anything, I was condensing it into my understanding.
The basis is not that brute force is better. The essence is most of us are not seeking an elegant solution to a problem. We are seeking a solution to a problem. Often, just getting that answer is all that matters. Finding a more concise way to get it is something I fully agree that someone should be trying to do. And, in the long term, it is a huge boon if it is found. For most tasks, though, the original solution is all that was needed.
Linked Observation: it is not very reliable in repetition.
I ran the sim a few times and got wildly different results.
- First time (about average) it stabalised (scopre >10,000) at Generation 18.
- The quickest stabalisation was at Generation 3
- sometimes it got to Generation 50 without stabalising.
Brute force will eventually get there, but I guess based on so few parameters it is easy to create a misleading weighting and difficult to un-learn that.
Right. But consider that if we just want a network that can play the perfect game of flappy bird, the correct answer is to find the parameters of a success and store those. Anything else is then just excess data. (This is calling the program that got the answer data. That happened to have been executed to get the perfect player.)
Amazing that this can be done in 300 lines of perfectly readable JS without any libraries. And the author apparently wrote it in 2 days. Great intro to genetic algorithms and reinforcement learning. Universities should teach like that.
Edit: Somebody has asked about the big picture how the thing works:
You use a neuronal network to decide at each step to flap the birds wings or not. That is the only output. The input is only the birds height (y-position) and the height of the next hole. (https://en.wikipedia.org/wiki/Feedforward_neural_network)
The question now is how to find the correct weights for the net.
That net is not trained in a traditional supervised way like with gradient descent (which would be more complicated). Instead it uses a genetic algorithms to find new weights for neuronal networks of future generations. That is use some of the best individuals. Some randomly generated ones and some created by breeding from existing individuals. (https://en.wikipedia.org/wiki/Genetic_algorithm)
And that's basically it in this case.