What Is Differentiable Programming?

Most importantly, differentiable programming is actually a shift opposite from the direction taken by deep learning; from increasingly heavily parameterised models to simpler ones that take more advantage of problem structure. For example, NLP models increasingly look more like classical grammar parsers with stack-augmented models, and one can even implement a Turing machine analog or a programming language interpreter in a differentiable way. Even the most mature areas of deep learning are not left out; after the convolution kernel, a natural next step for image models is the differentiable ray tracer.

Source: fluxml.ai