[SS 22] Deep Statistical Metric Learning (Rezaei)

Deep Metric learning aims to learn effective distance or similarity measures between arbitrary objects with the success of deep learning. The statistical deep metric learning goal is to learn statistical representation based on data distribution, density function and maps objects into an embedded space with more statistical information.  It’s an important topic in both natural language processing and computer vision and has been applied to a variety of tasks, including Grammar correction, and fine-grained image retrieval, object ranking, etc.

In this seminar, we will learn about the theory of deep metric learning and will review some state-of-the-art methods. We will offer different topics with different applications (i.e. NLP, CV, bioinformatics) for a variety of tasks (i.e. clustering, representation learning, density modeling, ranking, information retrieval, etc).  We plan to work on the extension of three categories:

1. Contrastive Approaches: Contrastive Loss, Triplet Loss, Improving the Triplet Loss 
2. Moving Away from Contrastive Approaches: Center Loss, Sphere Face 
3. State-of-the-art Approaches: CosFace, ArcFace, AdaCos Sub-Center ArcFace, ArcFace with Dynamic Margin.

As part of the seminar, you will also apply one of the frameworks to a given real-world problem. This means every participant will be asked to prepare an oral presentation about a current technique and to write up a reproducible case study of actual data analysis in a deep metric learning framework, in addition to peer-reviewing the (theoretical and practical) work of a colleague.

Recommended prerequisites: Deep learning; Python, PyTorch, TensorFlow, We would also hold the seminar in English and also allow students from other courses (especially DS students)