## Derin Öğrenme Coursera Notları (Deep Learning Coursera Notes)

Coursera’dan aldığım derin öğrenme kurslarından anahtar kelimeleri daha sonrası için hafızada tutmak amacıyla bu yazıda paylaşmak istiyorum.

NEURAL NETWORK AND DEEP LEARNING

• Supervised learning with neural network
• Binary classification (örn: logistic regression)
• Logistic regression, SVM -> traditional learning algorithms
• Cost function
• Derivatives
• Numpy, iPython, Jupyter
• Activation functions (Softmax, ReLU, leaky ReLU, tanh (hiberbolik tanjant), swish (a self-gated activation function))
• Forward / Back propagation
• Random initialization
• Shallow neural networks
• CNN (for image)
• Recurrent NN (Sequence data)
• Deep reinforcement learning
• Regression (Standart NN)
• Structured data – Database, Unstructured data – audio, image, text
• Tensorflow, Chainer, Theano, Pytorch
• Normalization
• Standart score
• T test
• Standardized moment
• Coefficient of variance
• Feature scaling (min-max)
• Circuit Theory
• Parameters
• W, b
• Hyperparameters
• learning rate
• #iterations
• #hidden layers
• #hidden units
• activation function
• momentum
• minibatch size
• regularization

IMPROVING DEEP NEURAL NETWORKS: HYPERPARAMETER TUNING, REGULARIZATION AND OPTIMIZATION

• Dataset -> Training / Dev (hold-out validation) / Test sets
• Büyük veri setleri için dağılım 98/1/1 olabilir. Geleneksel olarak 70/30 veya 60/20/20’dir.
• Bias / variance.
• high bias = underfitting
• bigger network (her zaman işe yarar)
• train longer (NN architecture search) (her zaman işe yaramaz)
• high variance = overfitting
• more data
• regularization
• L1 regularization
• L2 regularization (lambd) – daha çok tavsiye ve tercih edilir.
• Dropout regularization (keep prob)
• Data augmentation
• Early stopping
• NN architecture search
• Speed-up the training
• normalizing the inputs
• subtract mean
• normalize variance
• data vanishing / exploiding gradients
• weight initializion of deep networks
• xavier initialization
• HE initialization
• gradient checking -> backpropagation control
• dont use in training
• dtheta, dtheta approx.
• remember regularization
• does not work with dropout
• Optimization algorithms
• momentum
• RMSProp
• Mini batch
• Exponentially weighted averages
• Bias correction
• Learning rate decay
• The problem of local optima
• HYPERPARAMETER TUNING
• try random values
• confonets, resnets
• panda babysitting (sistem kaynakları kısıtlı ise) or baby fish (caviar) approach (değilse)
• batch normalization
• covariate shift
• softmax regression
• hardmax biggest 1, the others 0
• Frameworks
• Caffe/Caffe2
• CNTK
• DL4J
• Keras
• Lasagne
• mxnet
• Tensorflow
• Theano
• Torch

STRUCTURING MACHINE LEARNING PROJECTS

• Orthogonalization (eğitimin yeterince başarılı olması için gereklidir) (radyo ayarlama) (developer set (training)
• fit training set well in cost function
• bigger NN or better optimization algorithms
• fit dev. set well on cost function
• regularization or bigger training set
• fit test set well on cost function
• bigger dev set
• performs well in real world
• dev set is not set correctly, the cost function is not evaluating the right thing
• Single number evaluation metric
• P (precision) (toplam doğruluk, %95 kedidir)
• R (Recall) (kedilerin %90’ı doğru bilindi.
• F1 Score – average of precision and recall (F1 değeri yüksek olan daha başarılıdır)
• Satisficing and optimizing metric
• hangisi satisficing hangisi optimizing olacak.
• Train/dev/test sets distribution
• When to change dev/test sets and metrics
• Human level performance
• avoidable bias / bayes optimal error (best possible error)
• reducing bias/variance
• surprassing human-level performance
• ERRORS
• training
• variance
• more data
• regularization (lz, dropout, augmentation)
• NN architecture / hyperparameter search
• dev
• human-level errors
• avoidable bias
• train bigger model
• train longer
• train better optimization algorithms (momentum, RMSProb, Adam)
• NN architecture
• Hyperparameter search
• RNN/CNN
(30.04.2019 tarihinden itibaren toplam 18 kez, bugün 1 kez ziyaret edildi. )
Yorum Yazmak İster misiniz?

gaziantep escort

süperbetin giriş