There is no such thing as a free lunch



In computer science, we have a wide range of very different problems and many approaches to tackle them. We might for example try to find some optimal solution for a search problem, but it is unclear which algorithm one should use. Hence, it is only natural to think about algorithms that could always provide us the best solution, regardless of the problems subtleties.

However, even though a method might result in higher performance for one problem, this comes with the price of diminishing performance for other problems. This is formalized in the so-called 'No Free Lunch' theorems, which state that no single method works better than any other for all possible problems. In practice, it is necessary to carefully select a method for the individual problem at hand.

In other words, there is always a cost associated with selecting a method and unfortunately there is no such thing as a free lunch.