Are you interested in performing high-impact artificial intelligence research on embodied foundation models that will enable an autonomous robot to operate in an open world?
Progress in multimodal foundation models has been astonishing in the past few years and allow to equip robots with world knowledge of scenes, objects, and human activities. Robots should then be able to perceive and act upon the sensed world, be it that current solutions require data diversity, task circumstances, and the label vocabulary all to be pre-defined, stationary and controlled. As soon as these ‘closed world’ deep learning assumptions are broken, perceptual understanding suffers and oftentimes catastrophically. Hence, robots equipped with state-of-the-art multimodal perceptual skills will experience great difficulty generalizing to perception tasks in an open world where sensory and semantic conditions will differ considerably from those perceived during training.
Our key research question is: How to enable multimodal perception for robots that is robust to sensory and semantic shifts between training and operation conditions?
You will carry out research and development in the areas of embodied foundation models, deep machine learning and computer vision. Topics of interest are test-time generalization, embodied grounding, data scarcity and uncertainly modeling. The research is embedded in the VIS lab group at the University of Amsterdam, and you will actively collaborate within the OpenBots lab, that contains a team of five PhD students, two at the University of Amsterdam (this vacancy) and three at Delft University of Technology (focusing on planning and control). You will work three days a week at the University of Amsterdam and the other two days you will work with all other four PhDs at TNO and the Royal Netherlands Marechaussee where a physical lab environment with several land-robots is available. The project is carried out with supervisors from the Video and Image Sense Lab (Amsterdam) and the Cognitive Robotics Group (Delft). Students at UvA will be supervised by prof. dr. Cees Snoek and dr. ir. Gertjan Burghouts (TNO).
For more details on the vacancies pls check: https://vacatures.uva.nl/UvA/job/Two-PhD-Positions-on-Embodied-Foundation-Models/797275802/