Deep Learning is a state of the art algorithm in the digital economy.  We focus on Deep Learning to apply it in our daily lives

Deep learning is powerful and flexible enough to apply many problems to be solved. We focus on how to apply Deep Learning into real-life problems. There are many definitions about Deep Learning. In my view, this is the best.

"It’s a learning technology that works by loosely simulating the brain. Your brain and mine work by having massive amounts of neurons, jam-packed, talking to each other. And deep learning works by having a loose simulation of neurons — hundreds of thousands of millions of neurons — simulating the computer, talking to each other."(1)

There are three logics in order to understand the structure of deep learning

1. Logistic regression model as Neurons

Logistic regression is very useful for classification.  In deep learning,  logistic regression is used as a basic module.  First, data is input to a module.  Second, inner products of input data and parameters are obtained. Third, the results of the inner product are input to logistic function which provides probability of the event. If the events are more than one,  it is called multi class and the function is called "softmax" to provide probabilities of each event.  This function has also been researched as "discrete choice model" in Economics.

2. Cost function 

The criteria to evaluate the performance of deep learning models are needed. The criteria are provided by cost function.  "Negative log likelihood" is frequently used as a cost function.

3. Backpropagation method

In order to minimize cost function and optimize deep learning models,  derivative of the cost function with regard to parameters are needed. These derivatives are provided by back propagation.

(1) Andrew Ng, the Stanford computer scientist behind Google’s deep learning “Brain” team and now Baidu’s chief scientist.  Deep-Learning AI Is Taking Over Tech. What Is It?, Re/Code, Jule 15, 2015