Mike Erlihson

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Reviewer(s) Corner Read Recommendation from Mike & Avraham: highly recommended for representation learning fans, especially for those interested in contrastive learning Writing Clarity: average plus Prerequisite (prior knowledge): * Basic principles of representation learning and contrastive learning * Importance sampling Possible Practical Applications: Can be used to obtain powerful data representations from unlabeled datasets ---------------------------------------------------------

Computer Vision

Perceiver: General Perception with Iterative Attention

To overcome Transformers' squared complexity (w.r.t input length), the Perceiver article here offers a novel method to learn the QKV matrices. Check it out!

DeepNightLearners

Discriminator Rejection Sampling

A summary of a method to improve GAN generated images by exploiting accumulated "information" from the Discriminator during the GAN training process.

DeepNightLearners

Sharpness-Aware Minimization for Efficiently Improving Generalization

This very relevant article suggests an unusual method to improve neural network regularization abilities. I believe that this method has a great potential to enter the standard neural network training toolkit. I was also impressed by the detailed comparison to other methods.

DeepNightLearners

Representation learning via invariant causal mechanisms

This article suggests a method to construct augmentation resilient and styling invariant image representation in lower dimension (embeddings).

DeepNightLearners

Rethinking Attention with Performers

The article suggests a method to lower the Transformer's complexity to a linear order and proves all the arguments also in a rigorous form. The article is not easy to read, but luckily, to understand the main idea, the first 5-6 pages are more than enough.

DeepNightLearners

A causal view of compositional zero-shot recognition

One of the main challenges in zero-shot learning is enabling compositional generalization to the mode. This review is part of a series of reviews in Machine & Deep Learning that are originally published in Hebrew, aiming to make it accessible in a plain language under the name #DeepNightLearners.

DeepNightLearners

Supermasks in Superposition

The article suggests a brilliant training method, called SupSup, of a large neural network, which allows it to perform several different tasks.

Subscribe to Keep Current