Dear friends,
Last week, I wrote about the grand challenge of artificial general intelligence. Other scientific and engineering grand challenges inspire me as well. For example, fusion energy, extended lifespans, and space colonization have massive potential to remake civilization (for good or ill).
Despite their extreme uncertainty, such projects fill my mind with hopes and dreams. Fusion energy promises a safe, clean, unlimited source of electricity. The ability to harvest energy from the fusion of atoms could mitigate climate change and remake geopolitics by empowering all countries to become energy-independent.
Extended lifespans could enable people to accumulate greater wisdom. Of course, they could also concentrate wealth and power in the hands of the longest-lived individuals and create difficult demographic challenges. Purported longevity compounds like resveratrol have fallen short of their promise, but I’m excited by studies on the use of metformin and other compounds to lengthen lifespans. Space colonization that carries robots and, someday, humans to distant planets, solar systems, and ultimately galaxies would extend the future course of human history beyond the duration of Earth and into a practically unlimited future. Spacefaring technology would lead humanity into uncharted realms much like homo sapiens’ departure from Africa led to a global civilization.
Like artificial general intelligence, these grand challenges have motivated their share of overhyped startups, scorn from skeptics, and tireless enthusiasm from believers. Yet I hope to see progress in all of them within my lifetime. (If we manage to extend lifetimes, that could be a very long time.)
AI is not panacea. But as a general-purpose technology, it can be applied to these grand challenges and others. Whenever I’m interested in a topic, be it climate change or quantum computing, my background in AI makes it easier to strike up a fruitful conversation with domain experts. All of us in AI have tools that could be useful to them.
Keep learning! Andrew
NewsAI for PresidentA deepfake of South Korea’s new president helped propel him into office. What’s new: Yoon Suk-yeol, who won the country’s March 9 election, campaigned using videos that featured an AI-generated likeness of himself answering voters’ questions. No deception was involved; viewers were informed that they were watching a computer animation. How it works: Seoul-based DeepBrain AI created Yoon’s avatar using 20 hours of audio and video of the candidate captured in front of a green screen, totaling around 3,000 spoken sentences, according to France24.
Behind the news: The first known political use of deepfakes occurred in 2020, when Indian politician Manoj Tiwari altered a campaign video to show himself delivering the same message in various local languages. The technology has also fueled political scandals. In 2019, a Malaysian government minister said a video that captured him engaging in extramarital sex was a deepfake. Earlier that year, speculation that a video of Gabon’s president, Ali Bongo, was a deepfake had spurred an attempted coup. Why it matters: Yoon, who is known for his gruff, no-nonsense personality, created a digital double designed to resonate positively with the young voters who were deemed critical to his victory. While some critics dismissed the gambit, Yoon’s success suggests a bright future for campaign-sanctioned fakes tailored to appeal to particular groups. We’re thinking: A politician used a deepfake to make himself seem more authentic! How’s that for irony?
Know When to Fold ’EmLose too much money at Texas hold ’em, and you may get an AI-generated warning. What’s new: Casinos and gaming websites are using machine learning to flag gamblers who show signs of addictive behavior, The New York Times reported. How it works: Gambling businesses risk losing their licenses if they facilitate ruinous behavior. Moreover, they make more money on gamblers who pace themselves than those who lose their shirts. Denmark-based Mindway AI mitigates these risks by flagging worrisome behavior on the part of their customers. The system is mainly employed by online betting platforms, including Flutter Entertainment and Entain, but brick-and-mortar casinos have adopted the system as well.
Yes, but: Gambling addicts may not respond well to receiving automated messages telling them they have a problem, Brett Abarbanel, a gambling researcher at the University of Nevada Las Vegas, told The New York Times. Behind the news: Face recognition also plays a role in identifying problem gamblers. For instance, casinos in Macau have used the technology to identify high rollers and offer them perks. The city’s gambling authority stated that these systems were used only for security. Why it matters: As many as 10 million people suffer from compulsive gambling in the U.S. alone. Identifying problem gamblers helps combat the spiral of debt, substance abuse, and mental health issues that often follow. Of course, casinos benefit, too, if their patrons can remain solvent enough to keep pumping money back into the house. We’re thinking: For decades, the gambling industry has used data science to help casino operators. It’s heartening to see it applying AI to help its customers.
A MESSAGE FROM DEEPLEARNING.AILearn how to generate images using generative adversarial networks (GANs)! The Generative Adversarial Networks Specialization makes it easy to understand everything from foundational concepts to advanced techniques. Enroll today
Barnyard Sentiment AnalysisNeural networks may help farmers make sure their animals are happy. What’s new: Researchers led by Elodie Briefer and Ciara Sypherd at University of Copenhagen developed a system that interprets the moods behind a pig’s grunts and squeals. How it works: The authors trained convolutional neural networks to classify porcine expressions using a database of 7,414 vocal sounds made by animals engaged in 19 situations like feeding, fighting, running, or being led to a slaughterhouse.
Results: The models achieved 91.5 percent accuracy classifying the sentiment of calls and 81.5 percent identifying the situation. A method that classified calls without machine learning achieved 61.7 percent and 19.5 percent respectively. Behind the news: The noises an animal makes aren’t the only indication of its wellbeing, but they offer a window into its mental state.
Why it matters: The authors plan to develop a tool that would monitor hogs’ behavior and anticipate their needs. Science has shown that animals are capable of complex emotions, prompting countries like Australia and the United Kingdom to pass laws that protect livestock welfare. Systems that evaluate animals’ emotional states could help farms stay in regulatory compliance and make better homes for the creatures in their care, as well as reassure consumers that their food was produced humanely. We’re thinking: This work has awakened our interest in programming with EIEIO.
Who Needs Training?When you’re training a neural network, it takes a lot of computation to optimize its weights using an iterative algorithm like stochastic gradient descent. Wouldn’t it be great to compute the best parameter values in one pass? A new method takes a substantial step in that direction. What's new: Boris Knyazev and colleagues at Facebook developed Graph Hyper Network (GHN-2), a graph neural network that computed weights that enabled arbitrary neural network architectures to perform image recognition tasks. (A neural network that finds weights for another neural network is known as a hypernetwork.) GHN-2 improves on a similar hypernetwork, GHN-1, proposed by a different team. Key insights: GHN-1 learned based on how well a given architecture using generated weights performed the task. GHN-2 improved its predecessor’s performance by drawing on insights from training conventional neural networks:
GNN basics: A graph neural network processes datasets in the form of a graph made up of nodes connected by edges (say, customers connected to products they’ve purchased or research papers connected to other papers they cite). During execution, it uses a vanilla neural network to update the representation of each node based on the representations of neighboring nodes. How it works: GHN-2 consists of an embedding layer, a gated graph neural network, which uses a gated recurrent unit (a type of recurrent network layer) to update node representations, and a convolutional neural network. Its input is a neural network architecture in graph form, where each node represents a set of weights for an operation/layer such as convolution, pooling, or self-attention, and each edge is a connection from one operation/layer to the next. Its output is a set of weights for each operation/layer. The authors trained it to generate weights for classifying images in CIFAR-10 or ImageNet using a dataset of 1 million randomly generated neural network architectures composed of convolutional layers, pooling layers, self-attention layers, and so on.
Results: Architectures similar to those in the training set generally performed better using parameter values generated by GHN-2 than GHN-1. So did architectures that were wider, deeper, or more dense than those in the training set. Parameter values generated by GHN-2 yielded average CIFAR-10 accuracy of 66.9 percent versus GHN-1’s 51.4 percent. While GHN-2 outperformed GHN-1 on ImageNet, neither model produced great parameter values for that task. For instance, architectures similar to those in the training set and outfitted with parameter values from GHN-2 produced an average top-5 accuracy of 27.2 percent compared to GHN-1’s 17.2 percent. Why it matters: GHN-2 took only a fraction of a second to generate better-than-random parameter values, while training a ResNet-50 to convergence on ImageNet can take over one week on a 32GB Nvidia V100 GPU. (To be fair, after that week-plus of training, the ResNet-50’s accuracy can be 92.9 percent — a far better result.) We're thinking: The authors also found that initializing a model with GHN-2 boosted its accuracy after fine-tuning with a small amount of data. How much additional time did the initialization save compared to conventional initialization and fine-tuning?
Work With Andrew Ng
Frontend Desktop Application Engineer (Latin America): Landing AI seeks software development engineers to build scalable AI applications and deliver optimized inference software. Requirements include experience with JavaScript and open-source frameworks like React as well as cross-platform applications. Apply here
Data Analysts (Latin America): Factored seeks expert data analysts to analyze datasets for insights using descriptive modeling. Excellent English skills and a strong background in Python or R coding is required. Apply here
Data Engineer (Latin America): Factored is looking for top data engineers with experience with data structures and algorithms, operating systems, computer networks, and object-oriented programming. Experience with Python and excellent English skills are required. Apply here
Software Development Engineer (Latin America): Landing AI is looking for a software engineer with experience in best practices and proficiency in programming languages, as well as experience with end-to-end product development. In this role, you will help to design and develop infrastructure for machine learning services and deliver high-quality AI products to our clients. Apply here
Machine Learning Engineer: Workera is looking for an engineer to shape its product and create strategic advantages. You will build an intelligence layer critical to the value Workera offers its customers. Apply here
Data Scientist: Workera is looking for a data scientist to create unique value for users and enterprise clients. You will improve the company’s assessment capability, generate personalized learning plans, index content, and build other valuable applications. Apply here
Subscribe and view previous issues here.
Thoughts, suggestions, feedback? Please send to thebatch@deeplearning.ai. Avoid our newsletter ending up in your spam folder by adding our email address to your contacts list.
|