Dear friends,
In this letter, I’d like to address the serious matter of newcomers to AI sometimes experiencing imposter syndrome, where someone — regardless of their success in the field — wonders if they’re a fraud and really belong in the AI community. I want to make sure this doesn’t discourage you or anyone else.
An estimated 70 percent of people experience some form of imposter syndrome at some point. Many talented people have spoken publicly about this experience, including former Facebook COO Sheryl Sandberg, U.S. first lady Michelle Obama, actor Tom Hanks, and Atlassian co-CEO Mike Cannon-Brookes. It happens in our community even among accomplished people. If you’ve never experienced this yourself, that’s great! I hope you’ll join me in encouraging and welcoming everyone who wants to join our community. So if you, too, find parts of AI challenging, it’s okay. We’ve all been there. I guarantee that everyone who has published a seminal AI paper struggled with similar technical challenges at some point.
My three-year-old daughter (who can barely count to 12) regularly tries to teach things to my one-year-old son. No matter how far along you are — if you’re at least as knowledgeable as a three-year-old — you can encourage and lift up others behind you. Doing so will help you, too, as others behind you will recognize your expertise and also encourage you to keep developing. When you invite others to join the AI community, which I hope you will do, it also reduces any doubts that you are already one of us.
AI is such an important part of our world that I would like everyone who wants to be part of it to feel at home as a member of our community. Let’s work together to make it happen.
Your supporter and ally, Andrew
DeepLearning.AI ExclusiveFrom Outsider to EducatorWhen Jagriti Agrawal started her career, she felt hopelessly behind her peers. She caught up with help from friends and teachers. The experience led to work at NASA and co-founding her own education startup, as she explains in a new edition of our Breaking Into AI series. Read her story
NewsChipmaker Boosts AI as a ServiceNvidia, known for chips designed to process AI systems, is providing access to large language models. What’s new: Nvidia announced early access to NeMo LLM and BioNeMo, cloud-computing services that enable developers to generate text and biological sequences respectively, including methods that tune inputs — rather than the models themselves — to enable models trained on web data to work well with a particular user’s data and task without fine-tuning. Users can deploy a variety of models in the cloud, on-premises, or via an API.
Behind the news: Nvidia’s focus on prompt learning and biological applications differentiate it from other companies that provide large language models as a service.
Why it matters: Until recently, large language models were the province of organizations with the vast computational resources required to train and deploy them. Cloud services make these models available to a wide range of startups and researchers, dramatically increasing their potential to drive new developments and discoveries.
A MESSAGE FROM DEEPLEARNING.AIRobert Wydler was always drawn to AI. After 35 years in IT, he finally decided to pursue his passion by taking Andrew Ng’s Machine Learning course. Ready for a change? Enroll in the Machine Learning Specialization!
Panopticon Down UnderA state in Australia plans to outfit prisons with face recognition. What’s new: Corrective Services NSW, the government agency that operates nearly every prison in New South Wales, contracted the U.S.-based IT firm Unisys to replace a previous system, which required a fingerprint scan to identify people, with one that requires only that subjects pass before a camera, InnovationAus.com reported.
Yes, but: Samantha Floreani of Digital Rights Watch raised concerns that face recognition may exacerbate biases in the Australian corrective system, which incarcerates indigenous people disproportionately. Additionally, Floreani said that contracting to Unisys, a U.S.-based firm, raises questions about whether personal data on Australians will be transferred to another country and whether the data will be secure and handled properly. The Australian public, too, is wary. A 2021 poll found that 55 percent of Australians supported a moratorium on face recognition until stronger safeguards are in place. Why it matters: The flow of visitors, contractors, and prisoners into and out of correctional facilities creates opportunities for security breaches. Face recognition promises to help manage this traffic more safely. However, the technology, which is relatively new, largely unregulated, and developing rapidly, brings with it potential for abuse, mission creep, and other adverse consequences, especially in a high-stakes field like criminal justice.
Cookbook for Vision TransformersVision Transformers (ViTs) are overtaking convolutional neural networks (CNN) in many vision tasks, but procedures for training them are still tailored for CNNs. New research investigated how various training ingredients affect ViT performance. What's new: Hugo Touvron and colleagues at Meta and Sorbonne University formulated a new recipe for training ViTs. They call their third-generation approach Data Efficient Image Transformers (DeiT III). Key insight: The CNN and transformer architectures differ. For instance, when processing an image, a CNN works on one group of pixels at a time, while a transformer processes all pixels simultaneously. Moreover, while the computational cost of a CNN scales proportionally to input size, a transformer’s self-attention mechanism requires dramatically more processing as input size increases. Training recipes that take these differences — and other, less obvious ones — into account should impart better performance. How it works: The authors pretrained ViTs to classify images in ImageNet using various combinations of training data, data augmentation, and regularization. (They also experimented with variables such as weight decay, dropout, and type of optimizer, for which they didn’t describe results in detail.) They fine-tuned and tested on ImageNet.
Results: The authors’ approach substantially improved ViT performance. An 86 million-parameter ViT-B pretrained on ImageNet-21K and fine-tuned on ImageNet using the full recipe achieved 85.7 percent accuracy. Their cropping technique alone yielded 84.8 percent accuracy. In contrast, the same architecture trained on the same datasets using full-resolution examples augmented via RandAugment achieved 84.6 percent accuracy. Why it matters: Deep learning is evolving at a breakneck pace, and familiar hyperparameter choices may no longer be the most productive. This work is an early step toward updating for the transformer era recipes that were developed when CNNs ruled computer vision. We're thinking: The transformer architecture’s hunger for data makes it especially important to reconsider habits around data-related training procedures like augmentation and regularization.
Work With Andrew Ng
Senior Controller: Woebot Health seeks a controller to help drive the development of people, processes, technology, compliance, and reporting and ensure that data is available for decision-making. The ideal candidate has 10-plus years of experience. MBA and CPA preferred. Apply here
Product Marketing Manager: DeepLearning.AI seeks a product marketing manager who can bring its products to life across multiple channels and platforms including social, email, and the web. The ideal candidate is a creative self-starter who can work collaboratively and independently to execute new ideas and projects, thrives in a fast-paced environment, and has a passion for AI and/or education. Apply here
Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|