category: Deep Learning | Python Tutorial

Category: Deep Learning

Convolutional Deep Neural Networks

When it comes to neural networks, Convolutional Neural Networks or CNNs represent a major category to set-up image identification or classification. CNN image classification makes use of an input image, processes it and classifies it depending on specific features.

CNN is a class of deep neural network extensively used for computer vision or NLP. While the training process is in motion, the network`s building blocks are continuously modified in order to permit the network to achieve optimal performance, while classifying images and objects accordingly to the features specified.

Related Course:
Deep Learning for Computer Vision with Tensor Flow and Keras

Moreover, if the number of layers present in a neural network are boosted in order to deepen it, the complexity of the overall network is significantly increased, while permitting modification of more complicated functions. CNNs are usually encountered in computer vision and they act based on the idea of a moving filter. This filter is known as convolution and it is able to take into account specific neighbourhoods of nodes.

Bottom line, neural networks are sometimes considered to be basic Artificial Intelligence because of the fact that they start with a blank slate and work their way towards an accurate model. Even though their effectiveness is proved, some experts state neural networks are somehow inefficient in their approach to modelling.

Anyhow, algorithms like Hinton`s capsule networks use fewer sets of data to obtain an accurate model, which signifies that current research has the ability to resolve the basicness nature of deep learning, considered to be pure brute force.

Despite applications in other domains, DCNNs have remarkable results in image classification tasks, dominating various image classification challenges. Also, DCNNs determined researchers to thoroughly analyse both classification performance and computational features, which led to the discovery of different approaches to address them.

Deep learning networks undergo automatic feature extraction without human intervention, being a major difference between them and traditional machine-learning algorithms. Keep in mind that obtaining features from a set of data represents a task that can take data scientists years to achieve, while deep learning speed the process.

Recurrent Neural Networks

Recurrent Neural Networks provide an intriguing twist to common neural networks. A vanilla neural network acquires a fixed size vector as input and restricts its usage in scenarios that involve certain type of inputs with no preestablished size.

Related Course:
Deep Learning for Computer Vision with Tensor Flow and Keras

Recurent Neural Network

Furthermore, RNNs are able to remember the past, while the decisions made through an RNN are influenced by previously acquired knowledge. RNNs are able to take one or more input vectors, establish one or more output vectors, while the output is influenced by both weights applied on inputs and the context depicted by prior input/ output.

In other words, the input is able to offer a distinct output depending on the previous inputs featured in the series. Basically, a vanilla neural network stands for a fixed size input vector that is reconstructed into a fixed size output vector. In addition, the network becomes recurrent due to the repeated transformations done to a series of given input.

In a vanilla RNN, a long short-term memory network or LSTM provides extra gates and a cell state, which improves the RNN, along with addressing the issue of keeping or resetting the context.

Restricted Boltzman Networks

The Restricted Boltzman Machine is an algorithm invented by Geoffrey Hinton that is great for dimensionality reduction, classification, regression, collaborative filtering, feature learning and topic modelling.

RBMs represent shallow, two-layer neural nets that are able to set up building blocks of deep-belief networks. The primary layer of the RBM is called the visible layer, while the second is the hidden layer. However, the restriction of the RBM consists in the fact that there is no intra-layer communication, while each node that deals with the input makes stochastic decisions in regards to whether or not to forward the input.

Related Course:
Deep Learning for Computer Vision with Tensor Flow and Keras

Basically, RBM is similar to a two-way translator. Inputs are transformed into a set of numbers that encodes them, while a backward pass uses this set of numbers and depicts them into reconstructed inputs. Thus, the RBM is taught to re-establish the input with various weights and biases until the input is as close as it can be to the re-constructed data. This signifies that the data does not require to be labelled before the RBM due to the fact that it is automatically sorted out based on the weights and biases. RBM represents a feature identifier for neural nets able to depict existing patterns in the provided data.

Deep Belief Networks or DBNs

A deep belief network or DBN can be recognized as a set-up of restricted Boltzmann Machines for which every single RBM layer communicates with the previous and subsequent layers. However, the nodes of the mentioned layers are not able to communicate with each other laterally.

Furthermore, every layer in the deep belief network has a dual purpose, as it acts as the hidden layer to the nodes that come before it and as the visible layer to the nodes that come after it. An exception for this rule is represented by the first and final layers.

DBNs are great for recognizing, clustering and generating visual representations, video sequences or motion-captured data. In DBN each RBM learns the overall input, while the said training can be achieved in a shorter period of time by making use of GPUs, which provide accurate results compared to shallow nets.

Generative Adversarial Networks or GANs

The Generative Adversarial Networks represent a generative model, a set-up in which two distinct features depicted by neural networks are locked one against the other. The two main components are the generator and the discriminator, each one having specific roles in the framework.

The Generator establishes dissimulative samples of data and tries to fool the Discriminator, while the Discriminator attempts to identify real data samples from the dataset provided. Both the Generator and the Discriminator are neural networks meant to compete with each other in the training phase. Simply put, a GAN has two major parts as it follows:

  • The Discriminator is taught, while the Generator is latent. At this point, the network is pushed only forward and no back propagation is achieved. Thus, the Discriminator achieves real data and tries to correctly predict them as real. At the same time, the Discriminator is taught how to fake generated data from the Generator and establish if it can accurately predict them as fake.
  • The second part consists of the fact that after the Discriminator is trained through the established fake data from the Generator, predictions are obtained and used for training the Generator in order to fool the Discriminator

There are different types of GANs, while the most popular are Vanilla GAN, Conditional GAN, Deep Convolutional GAN and Super Resolution GAN. Anyhow, GANs potential is vast due to the fact that the network scan is able to mimic any distribution of data like music, speech or images.

Deep Nets

Deep learning networks are identified from the regular single layer neural networks through their depth. Depth represents the number of node layers by which the data has to enter in a multi-step operation of pattern recognition.

Hence, in deep-learning networks, every layer of nodes trains on different sets of features depending on the previous layer`s output. The deeper you advance into the neural net, the features identified will be more sophisticated due to the fact that they aggregate and reunite features from the previous layer. Thus, this is commonly known as feature hierarchy and it is characterized by the increasing complexity and abstraction of the features. It is the set-up that permits deep learning networks to handle vast, high-dimensional sets with billions of parameters that go through nonlinear functions.

Related Course:
Deep Learning for Computer Vision with Tensor Flow and Keras

What is even more interesting, is the fact that these neural nets are able to discover latent structures in the unstructured data or raw media such as photography, texts, video or audio recordings. Thus, the most important issue that deep learning deals with is processing and clustering the world`s raw media by identifying similarities and anomalies in such data that no human being has ever organized in a functional database.

As an example, deep learning can analyse a million images, identify their similarities and cluster depending on their particularities. This represents the basis of the popular smart photo albums that are able to establish different kind of photo albums depending on location, event or type of photography. Also, the same idea can be applied to different types of data due to the fact that deep learning can cluster raw text such as emails or news articles. This is what we call messaging filters used in customer relationship management.

Choosing a Deep Net

Well, the question that comes into the spotlight is how to opt for a specific deep net. The first step is to determine if the purpose is to build a classifier or if the goal is to identify patterns in the data. However, in order to get the patterns from a set of unlabelled data, an auto encoder or Restricted Boltzman Machine is required to be used, while the following steps should be considered:

  • If there is the case for analysing texts, parsing and name entity recognition, a recursive neural tensor network or RNTN is required
  • For language patterns that are used at character level, a recurrent net is to be used
  • For identifying images, a deep belief network or DBN is required
  • For recognizing objects, a RNTN is required
  • For speech recognition, a recurrent net is required

What is the difference between Deep Learning and Machine Learning?

Even though gaining understanding on how artificial intelligence works might be overwhelming, there are two main concepts that can aid you in gaining understanding of the process: machine learning and deep learning.

Both machine learning and deep learning are terms frequently used in relation to Artificial Intelligence. The core difference between the two is that machine learning represents the science of determining the machines to mimic humans without the need of programming, while deep learning is a subgroup of it.

Related course:


It is very important to understand the difference between the two concepts due to the fact that they are slightly different. Machine learning represents a concept in which algorithms get the data, learn from it and apply it in order to make informed decisions.

For machine learning, the algorithm is required to be told in what manner to establish an accurate prediction by offering it more data. In the case of deep learning, the algorithm is able to undergo the learning process through its own data processing. In some aspects, the whole process resembles how a human being would solve a problem and draw conclusions.

Machine learning employs automated algorithms which learn how to predict future decisions and determine functions depending on the data set they had, whereas deep learning is more complex. Deep learning interprets the data set, along with its relationships within the data by using neural networks in order to analyse the relevant data through various stages of data processing.

The multitude of algorithms used in machine learning are conducted by the analysts with the purpose to examine different variables in the data set. For deep learning, the algorithms are implemented and self-directed for proper data analysis. Moreover, for deep learning the output can consist of anything ranging from a score, free text or sound, while for machine learning the output is in most of the cases a numerical value such as a score or classification.

When it comes to the time required, deep learning needs more time to train as compared to machine learning. The issue is the fact that there are a multitude of parameters used in a deep learning algorithm. Machine learning is faster, ranging from a couple of seconds to a couple of hours.

Anyhow, both deep learning and machine learning have multiple purpose and are applied in a variety of real-life situations. Computer vision, information retrieval, marketing or medical diagnosis are all fields in which both are used to provide accurate data.