The following paragraphs summarize my experience in the 30 days of the challenge #ApaperADay from @nurture.ai. Each section contains the main insights obtained from each paper during the challenge. This post will be updated day by day according to the reading process.
Day 1
The authors explore a meta-learning approach to address learning problems when the labeling process of the data is costly. Particularly, this problem might be observed in medical applications, where the labeling process is expensive because the medical experts that need to label the data. In addition, such labels are not always available in large quantity.
They propose a model which aims to learning active-learning strategies using a pool-based setting, where the system observes all data and has to choose a subset of samples to label in a single shot learning approach. The proposed approach can be summarized in three steps: (1) the system receives an entire unsupervised set of examples, (2) a subset of examples is chosen and send to the oracle for labeling, and (3) the learning. This process is observed in the following Figure.
As results, the meta-learning method introduced by the authors outperforms two baselines approaches for active learning.
Day 2
Generative Adversarial Networks (GANs) (Figure 1) have become one of the best deep learning based generative models. Although GANs have been successfully applied in many scenarios; the training process requires a significant of hyper-parameter tuning, neural architecture engineering, an a non-trivial amount of tricks.
The authors review the current state of GAN architectures from a practical perspective. They reproduce the state-of-the-art architectures for several problems and go beyond by exploring several loss functions, regularization strategies, and normalization schemes for the discriminator. In addition, the authors show common issues in GANs training and provide pre-trained models for several applications in TensorFlow hub here.
The results indicate that in general, spectral normalization and gradient penalty regularization improve the quality of the model, especially in large architectures. In addition, the authors show that adding batch-normalization to the discriminator impacts the results in a bad way. In general, the experimental review performed by the authors is a suitable and fair benchmark for future research of GANs architectures.
Day 3
Today is the day for a NLP paper. This is a very nice paper that I enjoy to read.
The authors propose an effective transfer learning method that can be applied to many NLP tasks related to text classification. The proposed approach significantly outperforms the state-of-art in six classification tasks; even for small datasets. The pre-trained models and code from the authors is freely available here.
The “Universal Language Model Fine-tuning (ULMFit)” introduced by the authors enables a robust inductive transfer learning for any NLP task related to text classification. In addition, the authors proposed three novel techniques to retain previous knowledge and avoid catastrophic forgetting during the fine-tuning. Those new methods are the discriminative fine-tuning, slanted triangular learning rates, and gradual unfreezing.
The proposed strategy outperforms the state-of-art in six representative datasets for text classification, with an error reduction of 18–24%. The proposed model might be suitable to solve: (1) NLP problems for non-English languages, where training data for supervised pre training tasks is scarce; (2) new NLP tasks where no state-of-art architecture exists, such as language models for pathological applications; and (3) tasks with limited amounts of labeled data.
Final remarks:
The strategy of training each layer of the language model with a different learning rate (discriminative fine-tuning) is very interesting to capture and keep the information of each hide layer in an independent way. The gradual unfreezing is a very interesting approach to avoid catastrophic forgetting in transfer-learning problems.
Day 4
The paper proposes a novel method for sparse cross-domain correspondence with the aim to find correspondences between images from different semantic categories and that differ in appearance and shape. The introduced approach operates on hierarchies of deep features, extracted from the input images by a pre-trained CNN. From the feature maps obtained from the CNN, the authors search for Neural Best Buddies (NBB): pairs of neurons that are mutual nearest neighbors. The usefulness of the proposed method is demonstrated in a variety of applications, including cross-domain image alignment, creation of hybrid images, automatic image morphing, among others.
Day 5
There are applications where the interpretation of the hidden state of a neural network is a critical aspect that need to be addressed; especially where crucial decisions are performed based on the output of the network. The interpretation of the deep learning activations may be especially highly beneficial for medical image classification problems.
Several studies have been performed to address this problem; however there is still some issues that need to be solved. The authors from this paper proposed the Concept Activation Vectors (CAVs), which provide an interpretation of a neural net’s internal state in terms of human-friendly concepts.
The proposed method is a step toward creating a human-friendly linear interpretation of the internal state of a deep learning model, so that questions about model decisions may be answered in terms of natural high-level concepts.
Day 6
The ultimate goal of a reinforcement learning (RL) agent is to set its own goals and learn from experience with minimal outside intervention and manual engineering. The authors from this paper take a step toward this goal by designing a reinforcement learning framework that jointly learns representations of raw sensory inputs and policies that achieve arbitrary goals under this representation by practicing to reach self-specified random goals during training.
The method introduced by the authors combines an unsupervised representation learning and goal-conditioned policies. The representation learning of raw sensory inputs is learned by a latent variable model based on a variational autoencoder (VAE), which provides three main advantages: (1) it provides a more structured representation of the raw input data for RL; (2) it allows for sampling of new states, which can be used to set synthetic goals during training to allow the goal-conditioned policy to practice diverse behaviors, and (3) the learned representation provides a space where distances are more meaningful than the original space of observations, and can therefore provide well-shaped reward functions for RL.
By learning random goals sampled from the latent variable model, the goal-conditioned policy learns about the world and can be used to achieve new, user-specified goals at test-time. The results suggest that the introduced method highly improves the performance of image-based RL. The proposed algorithm can effectively learn policies for complex image-based tasks, and can be used to learn real-world robotic manipulation skills with raw image inputs. Videos of the method in simulated and real-world environments can be found at: https://sites.google.com/site/visualrlwithimaginedgoals/
Comentários