From Incremental to Exponential: Revolutionizing Generative AI

In our latest Leading with Data episode, Dr. Manish Gupta joins us with a global perspective, honed by leading teams across India, Australia, and the US. He previously led VideoKen, a pioneering video technology startup, and played a key role in directing research centers for Xerox and IBM in India. His impressive experience includes leading the development of system software for the Blue Gene/L supercomputer during his tenure as Senior Manager at the IBM T.J. Watson Research Center in Yorktown Heights, New York. Let’s look into the details of our conversation with Dr. Manish Gupta, exploring his insights and experiences in the field of AI.

You can listen to this episode of Leading with Data on popular platforms like Spotify, Google Podcasts, and Apple. Pick your favorite to enjoy the insightful content!

Key Insights from our Conversation with Manish Gupta

• The resurgence of deep learning and Transformer architecture has been pivotal in advancing AI capabilities across various domains.
• Large language models and self-supervision techniques have revolutionized AI by enabling models to generalize across tasks without task-specific training.
• Achieving AGI within the next decade is plausible, but ongoing challenges will continue to provide exciting research opportunities.
• Addressing the AI capability gap between mainstream and low-resource languages is essential for democratizing access to information.
• Academia and industry must collaborate to tackle fundamental AI challenges and develop more efficient architectures.
• Matrioska models offer a scalable and efficient way to deploy AI solutions that match available computational resources.
• Young professionals should pursue ambitious problems and view failures as learning opportunities for future success.
• Inclusive AI is crucial for leveraging AI to benefit every human on the planet, with a focus on language inclusivity, computational efficiency, and real-world applications.

Join our upcoming Leading with Data sessions for insightful discussions with AI and Data Science leaders!

Let’s look into the details of our conversation with Dr. Manish Gupta!

How did your early days in AI shape your journey to leading research at Google?

When I started at IBM Research in the US, my focus was on compilers and high-performance computing, not AI. However, upon my return to India, I was captivated by the impact of machine learning on real-world problems. This shift in focus led me to roles that increasingly centered around AI, culminating in my current position at Google, where I’m part of DeepMind, an organization dedicated to building AI responsibly to benefit humanity.

Reflecting on the evolution of AI, what were the key milestones that stood out to you?

The resurgence of artificial neural networks as deep learning marked a significant inflection point. The dramatic improvements in error rates for image classification signaled a broader trend where deep learning began to outperform more conventional ML approaches across various domains, including speech recognition and machine translation. The introduction of Transformer architecture and foundation models like BERT, which utilized self-supervision, further revolutionized the field by enabling models to excel at a wide range of tasks without task-specific training.

How did your perspective on AI evolve during this period?

Although I wasn’t initially a symbolic AI or neural network researcher, I quickly recognized the power of machine learning and deep learning. The advancements in these areas, especially the capabilities of large language models, were impressive. The ability of these models to generalize across tasks hinted at the potential for achieving artificial general intelligence (AGI).

What are your thoughts on the current trajectory of AI and the prospect of AGI?

We’re witnessing a convergence of multimodal models that understand text, speech, images, and videos. These models are becoming more robust and inclusive, though challenges remain. I’m optimistic that within the next decade, we’ll see systems with capabilities on par with humans across a broad range of tasks. However, as a researcher, I find the ongoing challenges exciting and believe there will always be complex problems to solve, even as we approach AGI.

How do you envision AI becoming accessible to every human on the planet?

There’s a significant gap in AI capabilities between mainstream languages like English and others, such as those spoken in India. Addressing this gap is crucial for democratizing access to information. Additionally, the computational intensity of large models presents a barrier to scaling AI globally. My team is actively working on making AI more inclusive and efficient to serve a larger number of users in a cost-effective and energy-efficient manner.

What are your views on the evolving roles of academia and industry in AI research?

I advocate for stronger academia-industry collaborations, which have improved significantly over the years. While industry has driven many AI advancements, academia plays a crucial role in addressing the fundamental challenges of current models and developing more efficient architectures. Both sectors are vital for the continued progress of AI.

Can you elaborate on the concept of Matrioska models and their potential impact?

Matrioska models, developed by my team, allow us to train large models that contain smaller, nested models within them. This approach enables us to deploy AI solutions that match the computational resources available or desired, offering a scalable and efficient way to utilize AI across various applications.

Reflecting on your career, what advice would you give to young professionals in AI?

Pursue ambitious problems that, if solved, could significantly impact the world. While there’s a place for incremental innovation, taking strategic risks and aiming for transformative breakthroughs can lead to more fulfilling and impactful careers. Embrace failures as learning opportunities, as they often pave the way for future successes.

What can attendees expect from your session at the upcoming Data Hack Summit?

I’ll be discussing the evolution of deep learning, the rise of foundation models, and the importance of inclusive AI. My focus will be on how we can leverage AI to benefit every human on the planet, addressing challenges in language inclusivity, computational efficiency, and applying AI to sectors like agriculture and public health.

Summing-up

In our engaging conversation with Dr. Manish Gupta, we uncovered pivotal advancements in AI, from deep learning to Transformer architecture, and discussed the path towards achieving AGI. Dr. Gupta emphasized the importance of inclusivity, collaboration between academia and industry, and the innovative potential of Matrioska models. His insights offer a compelling vision for the future of AI, highlighting both the challenges and exciting opportunities that lie ahead for professionals in this dynamic field.

For more engaging sessions on AI, data science, and GenAI, stay tuned with us on Leading with Data.

Check our upcoming sessions here.