Prof. Partha Pratim Das,
Professor of Computer Science, Ashoka University

Multimodal Food Computing: Challenges and Opportunities

Abstract: Food lies at the heart of human life, offering nourishment for the body and joy for the mind. However, the rise of diet-related health challenges – such as malnutrition, hypertension, diabetes, cardiovascular diseases, and anaemia – calls for informed and personalized dietary decisions. These decisions are influenced by factors like availability, affordability, cultural and personal preferences, and specific health conditions, creating a need for digital models, platforms, agents that unite diverse aspects of food and nutrition knowledge through Food Computing.

Food computing is inherently multimodal, integrating vision, smell, taste, touch, and language with computational methods to acquire and analyse diverse food-related data. Our perception of food is shaped by multiple sensory inputs and cognitive associations, making food a uniquely multimodal experience. Advances in AI, computer vision, natural language processing, and sensory modelling have enabled new ways to recognize, retrieve, recommend, predict, and monitor food, addressing key challenges in health, nutrition, sustainability, and food culture.

This presentation unveils the challenges and opportunities in this emerging area with the possible adoption of multimodal AI agents, deep and reinforcement learning, and large language models – underpinned by a curated food knowledge graph and a comprehensive food ontology.

Speaker Bio: Dr. Partha Pratim Das is a Professor at the Department of Computer Science and the Founding Director of the Center of Data Science and Analytics at Ashoka University. He has over 24 years’ experience in teaching and research at IIT Kharagpur, and about 13 years’ experience in Software Industry. He has worked extensively in Digital Geometry, Smart Software Engineering, and Digital Heritage for academic research; and EDA frontend automation and T & M tools in video technology as industrial products. His current interests are Food, Nutrition and Digital Health.

In Ashoka, Partha leads the initiative on Ashoka Datalake to host research data for climate, health, social and physical sciences and others, and anchors the project on “Multimodal Food Computing” in collaboration with Trivedi School of Biosciences at Ashoka and Institute of Future Health @ UCI.

Earlier, Prof. Partha had been involved in R & D and education on impact projects using huge volumes of integrated and curated data and tools of AI. From 2015 to 2022, he initiated and led the National Digital Library of India Project of Ministry of Education to create an open educational library having about 100 million+ free contents organized by educational needs of the users.

Prof. Partha has guided over a dozen doctoral theses and has published over 100 technical papers.