Högre seminarium: SignBot: generative AI for sign languages, Stockholms unversitet/KTH

SEMINARIUM
Datum: torsdag 20 februari 2025
Tid: 15:00 - 16:30
Plats: C307, Södra huset

Seminarietitel: SIGNBOT – Generative AI for Sign Language. Ett samarbete mellan Stockholms universitet och KTH. Talare: Johanna Mesch, Jonas Beskow, Simon Alexanderson, Anna Klezovich, Fredrik Malmberg.

Seminarium

Datum:

torsdag 20 februari 2025

Tid:

15.00 – 16.30

Plats:

C307, Södra huset

Man som tecknar och presentation av SignBot.

Foto: KTH

 

Abstract (engelska)

We are going to present to you the work we are doing in the ongoing project SignBot, dedicated to generative AI for sign languages at KTH, Speech, Music, and Hearing department.

In our studies so far we have been exploring representation learning for Swedish Sign Language (SSL) with an aim to use the learned representations for SSL generative modeling onto a 3D avatar.

The learned representations, or ”motion primitives”, were trained on sign language video data and evaluated against the phonological annotation from the SSL lexicon.

We also describe recording of our own motion capture dataset of SSL including detailed face and finger capture, how these recordings are mapped to high-quality sign language animation, and how this can be evaluated with native signers.

This data will later be used to train generative models, allowing the representations learnt from 2D to drive 3D sign language animation. In this talk we will discuss the recent advances in generative AI for sign language, our ongoing work and our future goals.

Om projektet SIGNBOT

Senast uppdaterad: 2025-02-16

Sidansvarig: Institutionen för lingvistik