Dear friends,
I’ve seen many new technologies go through a predictable process on their journey from idea to large scale adoption.
The data-centric AI movement is going through such a process. Data-centric AI is the growing discipline of systematically engineering the data needed to build successful AI systems. This contrasts with the model-centric approach, which focuses on inventing and tuning machine learning model architectures while holding the data fixed.
Experienced machine learning practitioners have been engineering data by hand for decades. Many have made learning algorithms work by improving the data — but, even when I was doing it years ago, I didn’t have the language to explain why I did things in a certain way. Now more and more teams are articulating principles for engineering data. I’m seeing exciting processes for spotting data inconsistencies, accelerating human labeling, applying data augmentation, and crowdsourcing more responsibly. Finally, just as TensorFlow and PyTorch made building neural networks more systematic, new tools are starting to emerge. Landing AI (where I am CEO) is building a platform for computer vision applications, and I expect many more tools to be built by different companies for different applications. They will enable teams to take what once was an ad hoc set of ideas and apply the right process at the right time.
Keep learning! Andrew
NewsWhich Drug Helps Your Depression?People seeking treatment for depression often experiment with different medications for months before finding one that works. Machine learning may remove some of the guesswork. What’s new: Deep learning can predict how patients will respond to two antidepressant medicines, according to a study led by Albert Montillo and Madhukar Trivedi at University of Texas Southwestern Medical Center. Key Insight: Patients with depression show various patterns of depressed brain activity in brain scans. At the same time, they vary in their reported responses to different drugs. Given brain scans of depressed people and their reports of effective treatment, a neural network can learn to match patients with medications likely to relieve their symptoms. How it works: The authors trained separate vanilla neural networks to predict the change in patients’ depression levels after treatment with each of two drugs as well as placebo.
Results: The authors evaluated their models on held-out data according to R2 value, a measure of performance in which 100 percent is perfect. The sertraline model achieved an R2 value of 48 percent. The bupropion model achieved 34 percent. Techniques that use brain scans to predict a patient’s response to drugs without deep learning have achieved R2 values around 15 percent, Montillo told The Batch. Why it matters: Millions of adults suffer from major depression, and one-third of those try at least three drugs before settling on one. Moreover, many doctors are influenced by outcomes they observe in a handful of patients and aren’t able to systematically analyze data from a large cohort. Reliable predictions about which medicines are likely to work best — even if they’re far from perfectly accurate — could make a difference. We’re thinking: Bringing this work into clinical practice would require training models to classify responses to many other antidepressants. The authors plan to apply their method to drugs beyond the two in this study, and we look forward to their progress.
AI Goes UndergroundComputer vision systems are surveying sewers for signs of decay and degradation. What’s new: A system from California startup SewerAI analyzes videos of underground pipes to prioritize those in need of repair. How it works: SewerAI’s computer vision system classifies defects like cracks, holes, displacements, tree roots, and incursions in videos taken by sewer-crawling robots and human inspectors.
Behind the news: AI is doing the dirty work for a growing number of companies.
Why it matters: Failed pipes can cause flooding, spread disease, and pollute water sources. In 2019, the American Society of Civil Engineers estimated the cost of shoring up the U.S. wastewater infrastructure at $129 billion — at least $81 billion more than lawmakers allocated in a recent law. By helping human inspectors prioritize repairs, computer vision could help stretch those dollars across more miles of pipe. We’re thinking: Would we rather let a robot inspect sludge-filled pipes than do it ourselves? Sewer we would!
A MESSAGE FROM DEEPLEARNING.AIStart our Machine Learning Engineering for Production (MLOps) Specialization today! Learn how to design, build, and maintain integrated systems using well-established tools and methods. Enroll now
Everyone Has a VoiceGoogle is improving speech recognition for people who have conditions that affect their ability to pronounce words. What’s new: Project Relate is an Android app that offers a personalized speech recognition model for people whose speech is impaired or otherwise atypical. It’s designed to understand, transcribe, and synthesize speech for both person-to-person and person-to-machine interactions. How it works: Researchers trained Relate using over a million speech samples collected from people with conditions including amyotrophic lateral sclerosis, cerebral palsy, Down syndrome, Parkinson’s disease, stroke, or traumatic brain injury. Users can fine-tune the system to their own voices by speaking 500 phrases.
Behind the news: Recognizing the need to make their products more inclusive, tech companies have launched initiatives to make apps more accessible.
Why it matters: People whose speech is atypical can be excluded from social interactions, have trouble communicating when they need help, and experience difficulty using voice-activated devices. Technology that lets them be heard could make their lives richer, safer, and more engaging. We’re thinking: Speech recognition is a convenience for most people, but for those with unusual speech patterns, it could be a lifeline.
A Deeper Look at GraphsNeural networks designed to process datasets in the form of a graph — a collection of nodes connected by edges — have delivered nearly state-of-the-art results with only a handful of layers. This capability raises the question: Do deeper graph neural networks have any advantage? New research shows that they do. What’s new: Ravichandra Addanki and colleagues at DeepMind probed the impact of depth on the performance of graph neural networks. GNN basics: A graph neural network (GNN) operates on graphs that link, for instance, customers to products they've purchased, papers to the other papers they cite, or pixels adjacent to one another in an image. A GNN typically represents nodes and edges as vectors and updates them iteratively based on the states of neighboring nodes and edges. Some GNNs represent an entire graph as a vector and update it according to the representations of nodes and edges. Key insight: Previous work found that adding a few layers to a shallow GNN barely improved performance. That study used graphs that comprised hundreds of thousands of nodes and edges. Since then, graphs have emerged with hundreds of millions of nodes and edges. Deeper GNNs may achieve superior performance on these larger datasets. How it works: The authors built GNNs up to more than 100 layers deep, including an encoder (a vanilla neural network), a graph network made up of message-passing blocks (each a trio of vanilla neural networks), and a decoder (another vanilla neural network). Among other experiments, they trained a GNN on 4 million graphs of molecules, in which nodes are atoms and edges are bonds between them, to estimate a particular key property called the HOMO-LUMO gap. (This property helps determine a molecule’s behavior in the presence of light, electricity, and other chemicals.)
Results: The authors tested GNNs with different numbers of message-passing blocks. Performance on the validation set improved progressively with more message-passing blocks up to 32 — 104 layers total — but showed no benefit beyond that depth. A version with 8 message-passing blocks achieved ~0.128 mean absolute error, one with 16 achieved ~0.124 mean absolute error, and one with 32 achieved ~0.121 mean absolute error. Why it matters: Not all types of data can be represented easily as an image or text — consider a social network — but almost all can be represented as a graph. This suggests that deep GNNs could prove useful in solving a wide variety of problems. We’re thinking: CNNs and RNNs have become more powerful with increasing depth. GNNs may have a lot of room to grow.
Work With Andrew Ng
Software Development Engineer (LatAm): Landing AI seeks a software development engineer to build scalable AI applications and deliver optimized inference software. A strong background in Docker, Kubernetes, infrastructure, network security, or cloud-based development is preferred. Apply here
Frontend Desktop Application Engineer (LatAm): Landing AI is looking for a software development engineer to develop AI applications for clients in manufacturing, agriculture, and healthcare. Proficiency in programming languages and experience with end-to-end product development is preferred. Apply here
Part-time Machine Learning Instructor: FourthBrain is seeking machine learning practitioners or educators to teach cohort-based programs in practical machine learning. Apply here
Head of Digital Marketing: Factored seeks a highly experienced digital marketer with a strong knowledge of paid media, search engine optimization, campaign management, and marketing automation. Experience leading a marketing team and impeccable written and spoken English is required. Apply here
Sales Development Representative (North America): Landing AI is looking for a salesperson to generate new business opportunities through calls, strategic preparation, and delivering against quota. Experience with inside sales and enterprise products and a proven track record of achieving corporate quotas is preferred. Apply here
Machine Learning Engineer (North America): Landing AI is searching for a machine learning engineer to work with internal and external engineers on novel models for customers. A solid background in machine learning and deep learning with a proven ability to implement, debug, and deploy machine learning models is a must Apply here
Technical Writer: Landing AI seeks a writer to own the product education and documentation effort. The ideal candidate is self-motivated, can learn new tools and Landing AI applications quickly, and communicates effectively. Apply here
Director of Machine Learning: Landing AI seeks a machine learning director to define the vision for its products. This person will build and lead an effective machine learning team to execute projects in collaboration with other teams. Apply here
Frontend Desktop Application Engineer (LatAm): Landing AI is looking for a software development engineer to develop AI applications for clients in manufacturing, agriculture, and healthcare. Proficiency in programming languages and experience with end-to-end product development is preferred. Apply here
Community and Events Marketing Manager: DeepLearning.AI seeks a community and events marketing manager. The ideal candidate is a talented leader, communicator, and creative producer who is ready to create world-class events that keep the community connected and engaged with each other. Apply here
Digital Marketing Manager: DeepLearning.AI is looking for a digital marketing manager to oversee digital marketing campaigns, manage data and analytics, and optimize workflows and processes. The ideal candidate is a strong project manager, communicator, and technical wizard who can work closely with the content, social, events, and community teams. Apply here
Data Engineer (LatAm): Factored is looking for top data engineers with experience in data structures and algorithms, operating systems, computer networks, and object-oriented programming. Candidates must have experience with Python and excellent English-language skills. Apply here
Senior Technical Program Manager: Landing AI is looking for a program manager to bridge our team and business partners in executing engineering programs. The ideal candidate has a strong customer relationship management background, three years of experience in a direct program management position, and two years of experience in a technical role. Apply here Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|