The Restricted Boltzman Machine is an algorithm invented by Geoffrey Hinton that is great for dimensionality reduction, classification, regression, collaborative filtering, feature learning and topic modelling.
RBMs represent shallow, two-layer neural nets that are able to set up building blocks of deep-belief networks. The primary layer of the RBM is called the visible layer, while the second is the hidden layer. However, the restriction of the RBM consists in the fact that there is no intra-layer communication, while each node that deals with the input makes stochastic decisions in regards to whether or not to forward the input.
Basically, RBM is similar to a two-way translator. Inputs are transformed into a set of numbers that encodes them, while a backward pass uses this set of numbers and depicts them into reconstructed inputs. Thus, the RBM is taught to re-establish the input with various weights and biases until the input is as close as it can be to the re-constructed data. This signifies that the data does not require to be labelled before the RBM due to the fact that it is automatically sorted out based on the weights and biases. RBM represents a feature identifier for neural nets able to depict existing patterns in the provided data.
A deep belief network or DBN can be recognized as a set-up of restricted Boltzmann Machines for which every single RBM layer communicates with the previous and subsequent layers. However, the nodes of the mentioned layers are not able to communicate with each other laterally.
Furthermore, every layer in the deep belief network has a dual purpose, as it acts as the hidden layer to the nodes that come before it and as the visible layer to the nodes that come after it. An exception for this rule is represented by the first and final layers.
DBNs are great for recognizing, clustering and generating visual representations, video sequences or motion-captured data. In DBN each RBM learns the overall input, while the said training can be achieved in a shorter period of time by making use of GPUs, which provide accurate results compared to shallow nets.
The Generative Adversarial Networks represent a generative model, a set-up in which two distinct features depicted by neural networks are locked one against the other. The two main components are the generator and the discriminator, each one having specific roles in the framework.
The Generator establishes dissimulative samples of data and tries to fool the Discriminator, while the Discriminator attempts to identify real data samples from the dataset provided. Both the Generator and the Discriminator are neural networks meant to compete with each other in the training phase. Simply put, a GAN has two major parts as it follows:
- The Discriminator is taught, while the Generator is latent. At this point, the network is pushed only forward and no back propagation is achieved. Thus, the Discriminator achieves real data and tries to correctly predict them as real. At the same time, the Discriminator is taught how to fake generated data from the Generator and establish if it can accurately predict them as fake.
- The second part consists of the fact that after the Discriminator is trained through the established fake data from the Generator, predictions are obtained and used for training the Generator in order to fool the Discriminator
There are different types of GANs, while the most popular are Vanilla GAN, Conditional GAN, Deep Convolutional GAN and Super Resolution GAN. Anyhow, GANs potential is vast due to the fact that the network scan is able to mimic any distribution of data like music, speech or images.