Dear friends,
I’m thrilled to announce the NeurIPS Data-Centric AI Workshop, which will be held on December 14, 2021. You may have heard me speak about data-centric AI, in which we systematically engineer the data that feeds heard me speak about data-centric AI, in which we systematically engineer the data that feeds learning algorithms. This workshop is a chance to delve more deeply into the subject.
Indeed, even deep learning once was a niche topic at NeurIPS, and my friends and I organized workshops to share ideas and build momentum. My team at Landing AI (which is hiring!) is inventing data-centric algorithms for image data as part of an MLOps platform for computer vision. I’d love to see hundreds or thousands more groups working on data-centric algorithms.
Open questions include:
The workshop is accepting research paper submissions that address such issues until September 30, 2021. Please check out the website for details.
Keep learning! Andrew
NewsGetting a Jump on Climate ChangeStartups are predicting how climate change will affect global commerce. What’s new: Companies that specialize in climate analytics are training neural networks to help businesses manage risks posed by a warming globe, The Wall Street Journal reported. Changes in the air: These young companies model interactions among environmental data and factors such as commodity prices, consumption patterns, and import/export data. They sell the resulting insights to corporate customers who are concerned about the impact of climate change on their ability to buy goods and raw materials.
Behind the news: Corporations are waking up to the hazards posed by climate change to their own well-being.
Why it matters: This year’s run of record-breaking wildfires, floods, and freezes are a preview of what to expect in a warmer world, according to the latest International Panel on Climate Change report. AI-powered forecasts can help businesses protect assets and revenue — and the rest of us prepare for further impacts to come. We’re thinking: By calculating the costs of climate disaster, AI can make the very real danger posed by atmospheric carbon emissions feel as urgent as it is.
AI Sales Closing In on $500 BillionA new report projects a rosy future for the AI industry. What’s new: A study from market research firm IDC estimates that global revenues for AI software, hardware, and services will reach $341.8 billion in 2021 — up from an estimated $156.5 billion last year — and will break $500 billion by 2024. The study reflects interviews, distribution statistics, financial reports, and other data from over 700 AI companies around the world. What they found: The AI industry’s annual growth rate is expected to exceed 18.8 percent next year. The analysis breaks up that growth into three broad categories. Some of the most important findings:
Behind the news: IDC’s most recent predictions are in line with their previous report, published in February, and jibe with research from MIT Technology Review. Why it matters: In the AI world — as in other high-tech sectors — it’s often difficult to discern real growth potential from gossip-fueled hype. Research reports that provide granular insights are a crucial tool for business leaders and investors who aim to capitalize on this industry, not to mention machine learning engineers who are plotting a career. We’re thinking: We’ve seen market research reports that later proved right and many that later proved dead wrong. We hope this is one of the former!
A MESSAGE FROM DEEPLEARNING.AILearn how to design machine learning production systems end-to-end in “Deploying Machine Learning Models in Production,” Course 4 of the Machine Learning Engineering for Production (MLOps) Specialization on Coursera! Enroll now Perceptrons Are All You NeedThe paper that introduced the transformer famously declared, “Attention is all you need.” To the contrary, new work shows you may not need transformer-style attention at all. What’s new: Hanxiao Liu and colleagues at Google Brain developed the gated multi-layer perceptron (gMLP), a simple architecture that performed some language and vision tasks as well as transformers. Key insight: A transformer processes input sequences using both a vanilla neural network, often called a multi-layer perceptron, and a self-attention mechanism. The vanilla neural network works on relationships between each element within the vector representation of a given token — say, a word in text or pixel in an image — while self-attention learns the relationships between each token in a sequence. However, the vanilla neural network also can do this job if the sequence length is fixed. The authors reassigned attention’s role to the vanilla neural network by fixing the sequence length and adding a gating unit to filter out the least important parts of the sequence. How it works: To evaluate gMLP in a language application, the authors pretrained it to predict missing words in the English version of the text database C4 and fine-tuned it to classify positive and negative sentiment expressed by excerpts from movie reviews in SST-2. For vision, they trained it on ImageNet using image patches as tokens.
Results: In tests, gMLP performed roughly as well as the popular transformer-based language model BERT. The authors compared the performance on C4 of comparably sized, pretrained (but not fine-tuned) models. gMLP achieved 4.28 perplexity, which measures a model’s ability to predict words in a test set (smaller is better), while BERT achieved 4.17 perplexity. On SST-2, gMLP achieved 94.2 percent accuracy, while BERT achieved 93.8 percent accuracy. The authors’ approach performed similarly well in image classification after training on ImageNet. gMLP achieved 81.6 percent accuracy compared to a DeiT-B’s 81.8 percent accuracy. Why it matters: This model, along with other recent work from Google Brain, bolsters the idea that alternatives based on old-school architectures can approach or exceed the performance of newfangled techniques like self-attention. We’re thinking: When someone invents a model that does away with attention, we pay attention!
Solar SystemAstronomers may use deep learning to keep the sun in focus. Key insight: Although the sun is a writhing ball of fiery plasma, patterns across its surface correlate with its brightness. A neural network can learn to associate these patterns with their characteristic brightness, so its output can be used to recalibrate equipment that monitors Earth’s nearest star.
Results: In tests using images taken by uncalibrated equipment, the model outperformed a baseline method that didn’t involve machine learning. Defining success as a prediction within 10 percent of the actual degree of dimming, the authors obtained 77 percent mean success across all wavelengths. The baseline achieved 43 percent mean success.
Work With Andrew Ng
Head of Engineering: Zest is looking for a Head of Engineering with experience in developing and shipping consumer-facing mobile apps for iOS and Android, to provide thought leadership and establish a technical vision for the team. Apply Here
Data Engineer (LatAm): Factored is looking for top data engineers with experience in data structures and algorithms, operating systems, computer networks, and object-oriented programming. You must have experience with Python and excellent skills in English. Apply here Software Development Engineer: Landing AI seeks software development engineers to build scalable AI applications and deliver optimized inference software. A strong background in Docker, Kubernetes, infrastructure, network security, or cloud-based development is preferred. Apply in North America or Latin America. Machine Learning Engineer (Customer Facing): Landing AI is looking for a machine learning engineer to work with internal and external engineers on novel models for customers. A solid background in machine learning and deep learning with proven ability to implement, debug, and deploy machine learning models is a must. Apply here Sales Development Representative (North America): Landing AI is looking for a salesperson to generate new business opportunities through calls, strategic preparation, and delivering against quota. Experience with inside sales and enterprise products and a proven track record of achieving corporate quotas is preferred. Apply here Software Engineers (Remote): Workera, a precision upskilling company that enables individuals and organizations to identify, measure, interpret, and develop AI skills, is looking for software engineers of all levels. You’ll own the mission-critical effort of implementing and deploying innovative learning technologies. Apply here Solutions Architect: Workera is looking for a solutions architect to empower its go-to-market team, create a streamlined sales-enabling environment, and accelerate business opportunities. Apply here Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|