A Meta Lesson

I think that Sutton and Brooks both take clear positions with strong examples, my main contention would be that Brooks discounts using learning and search to improve learning itself and so misses the deeper implications of Sutton’s position. But you see what happened, in the past we hand-crafted features, now we hand-craft architectures (A point Brooks himself makes) this is exactly what Rich Sutton is critical of; we should be always seeking to do more with learning and search. Architecture search is relatively unsophisticated at the moment, but it will hopefully give us something better than just throwing countless PhD students at the problem (an algorithm I call… graduate descent)

Brooks goes on to criticise the need for massive data sets in Deep Learning, this is indeed a fair criticism but totally mischaracterises Sutton; he doesn’t even use the word ‘data’ once in his essay.

Source: andy.kitchen