Nvidia's smart driving empire

Nvidia's smart driving empire

Recently, there has been a significant development in the autonomous driving industry.

Firstly, Apple Inc. has put on hold and canceled all development plans for its autonomous electric vehicles. This project, which cost over a decade and more than a billion dollars, has been declared as sinking into the iceberg. Reports suggest that the project will begin to scale down gradually, and many members of the automotive team will be reassigned to the artificial intelligence department.

While Apple's autonomous driving is undergoing a dramatic "pain of division," NVIDIA's intelligent driving team is quietly "recruiting troops." Luo Qi, who was responsible for the overall software architecture of the vehicle-side for Baidu's intelligent driving L2+ business, as well as the control and vehicle interaction technology, has joined NVIDIA's automotive division as an engineering director, in charge of prediction, planning, and control. His reporting person is Wu Xinzhou, who recently joined NVIDIA's automotive department.

Advertisement

The public's focus on NVIDIA's automotive business may be more on automotive chips, and the last time it was in the spotlight was when Wu Xinzhou, the key figure in Xiaopeng's intelligent driving, joined. In fact, NVIDIA's intelligent driving business is not new, but the results of the team's projects seem to be little known. So, how has NVIDIA's intelligent driving business developed?

01

The Strong NVIDIA Automotive Business

NVIDIA has three automotive platforms, namely NVIDIA DRIVE® Infrastructure, NVIDIA DRIVE AGX®, NVIDIA DRIVE Concierge, and NVIDIA DRIVE Hyperion®.

NVIDIA DRIVE® Infrastructure is a complete workflow platform for data extraction, maintenance, marking, and training, and can also validate data through simulation. The NVIDIA DGX® system provides the necessary computing power for large-scale training and optimization of deep neural network models. NVIDIA DRIVE Constellation® enables physics-based simulation on an open hardware-in-the-loop platform, allowing for testing and validation of autonomous vehicles before they hit the road.The NVIDIA DRIVE AGX™ platform, with its scalable and software-defined features, offers advanced performance to help autonomous vehicles process large amounts of sensor data and make real-time driving decisions. The open NVIDIA DRIVE software stack also assists developers in building perception, mapping, planning, and driver monitoring functions using redundant and diverse deep neural networks (DNNs). Through continuous iteration and over-the-air updates, the platform becomes increasingly powerful.

With NVIDIA DRIVE Concierge, vehicle occupants can utilize a range of intelligent services based on NVIDIA DRIVE IX and NVIDIA Omniverse™ ACE (Avatar Cloud Engine). NVIDIA DRIVE Chauffeur is an AI-assisted driving platform based on the NVIDIA DRIVE AV SDK that enables point-to-point driving. If you prefer to drive yourself, it also provides active safety features that intervene in dangerous situations.

NVIDIA DRIVE Hyperion™ is a complete development platform and reference architecture for designing autonomous vehicles. This architecture accelerates development, testing, and validation by integrating AI computing based on NVIDIA Orin™ with a comprehensive sensor suite. DRIVE Hyperion features a full software stack for autonomous driving (DRIVE AV), as well as driver monitoring and visualization functions that can be updated wirelessly (DRIVE IX). This allows for the addition of new features and functions throughout the vehicle's lifecycle.

From the distribution of such platforms, it can be seen that NVIDIA has already accumulated a certain foundation in software support for intelligent driving. As a leader in computing power, NVIDIA is also not lacking in hardware support. However, why hasn't NVIDIA been able to "dominate" the autonomous driving industry yet? Some industry insiders told ICVIEWS that part of the reason for the inability to implement is that NVIDIA does not understand the automotive business well enough and lacks a key leading role. This has led to a situation where, despite having technology in the automotive field, NVIDIA's intelligent driving team remains "weak."

02

"Weak" NVIDIA Intelligent Driving Team

 

Although automotive is listed as one of NVIDIA's three major businesses (data centers, gaming, automotive), its revenue share is far from the other two. NVIDIA's financial report for the fourth quarter of 2023 shows that automotive business revenue was $281 million, a sequential increase of 8%, a year-over-year decrease of 10%, accounting for 1.2% of total revenue.

Within the automotive business, the autonomous driving team is even more "weak."

In 2015, NVIDIA began the development of autonomous driving solutions. As an absolute leader in the chip industry, NVIDIA's intelligent driving made a direct impact on the industry's top users, reaching cooperation intentions with several major overseas car manufacturers such as Mercedes-Benz and Jaguar Land Rover.In June 2020, NVIDIA and Mercedes-Benz officially announced their collaboration, with NVIDIA providing the AI software architecture for Mercedes-Benz's next-generation vehicles, including autonomous driving software solutions and smart cockpits. Following the partnership, executives from Mercedes-Benz frequently appeared at NVIDIA's GTC (developer) conferences. Jen-Hsun Huang, NVIDIA's CEO, also made several high-profile appearances to endorse Mercedes-Benz, often featured in promotional materials for the new S-Class models.

The collaboration between NVIDIA and Mercedes-Benz is not based on engineering fees or IP licensing fees, but rather, in addition to the basic R&D costs, they share profits based on the sales volume of Mercedes-Benz's new products. This partnership represents a new attempt in the history of automotive development, as traditional OEMs have not typically collaborated with suppliers on a profit-sharing model based on sales volume. However, NVIDIA has not demonstrated the expected capabilities. Reports suggest that during a demonstration of the assisted driving capabilities to Mercedes-Benz, NVIDIA's intelligent driving team failed to successfully execute a side reverse maneuver even after several attempts. This has led to a less stable partnership, with Mercedes-Benz at one point requesting the introduction of new suppliers.

NVIDIA's intelligent driving team may need a "savior," and this person could be Xinzhu Wu.

Xinzhu Wu graduated from Tsinghua University with a bachelor's degree and then went on to pursue his Ph.D. in Electrical Engineering at the University of Illinois at Urbana-Champaign (UIUC), which he obtained in 2004. He then joined the telecommunications startup Flarion, where he worked as a technical engineer primarily responsible for the design and simulation of PHY/MAC systems. At Flarion, he was involved in the development of multiple technologies and later joined Qualcomm when Flarion was acquired. After accumulating 13 years of technical experience at Qualcomm, he became an engineering director in 2015, leading the development of Qualcomm's autonomous driving technologies, focusing on high-precision maps, cameras, radar, and deep learning.

With the addition of such a key figure, is there hope for the technically advanced NVIDIA to take off again in the automotive race?

03

NVIDIA's Ambition: $30 Billion

Semiconductor chips have always been NVIDIA's competitive edge.

Even without dominating the field, NVIDIA has never been an unknown entity in the automotive industry. In 2019, NVIDIA launched its new generation of autonomous driving chip, the Orin, which attracted all L4 and above autonomous driving companies in China as buyers. Today, NVIDIA's list of automotive partners includes almost all "star" car manufacturers. For example: Mercedes-Benz, Jaguar, Land Rover, Volvo, Hyundai, BYD, Polestar, NIO, and so on.Compared to other companies, NVIDIA possesses a vast algorithmic ecosystem barrier. As early as 2015, the idea of adopting an "end-to-end" neural network for autonomous driving was proposed within NVIDIA, and a preliminary demo was already available. Tesla, as a leading enterprise in autonomous driving, has brought this end-to-end AI system for autonomous driving into reality. Elon Musk live-streamed the testing of Tesla's FSD Beta V12 on the streets of Silicon Valley, attracting over 40 million viewers online. Although NVIDIA and Tesla implement "end-to-end" solutions differently, the underlying goal is the same: to mimic biological intelligence, allowing the autonomous driving system to make decisions unbound by rules. The fact that NVIDIA, considered an "outsider" at the time, proposed such an idea in 2015 is a testament to its technical strength.

Moreover, NVIDIA's strategic positioning in automotive-related platforms, combined with its own hardware chips, has the potential to significantly reshape the market for autonomous driving.

Some lidar companies have indicated that it is easier to access all their lidar plug-ins on NVIDIA's Drive computing platform. If companies are developing solutions using platforms like Xavier (their previous generation of autonomous driving computing chips), their perception and other software can be compatible with any of their computing units.

NVIDIA is entering the automotive market in a "silent yet pervasive" manner—making it easier for automotive component manufacturers' software to run on their hardware. NVIDIA's full-stack system solution provides more insightful understanding for the implementation of autonomous driving. The full-stack system covers various aspects including hardware, software, and ecosystem, giving NVIDIA a stronger competitive edge in the field of autonomous driving. In addition, NVIDIA is also committed to building a robust ecosystem to allow more autonomous driving sensor hardware and software capabilities to take root and grow within this system.

Although current revenues are not high, NVIDIA's founder, Jen-Hsun Huang, is very optimistic about the prospects of the automotive business. Huang once stated that NVIDIA's future revenue could reach 1000 billion US dollars, with the automotive business accounting for 30%. This implies that NVIDIA plans to achieve 30 billion in revenue in the automotive sector.

Since Wu Xinzhu, the former vice president of autonomous driving at XPeng Motors, joined NVIDIA, dozens of people including Liu Langgechuan, the former AI leader of XPeng's autonomous driving, Parixit Aghera, the former VP of XPeng's North American team, Han Feng, the director of multimodal perception fusion algorithms, and Houman Tavakoli, the software architecture director, have joined or are about to join NVIDIA's intelligent driving team.

At the end of 2023, NVIDIA's China division opened recruitment for five departments, including the autonomous driving software group, the autonomous driving platform group, the system integration and testing group, the mapping and simulation group, and the product group, with a total of 25 positions. As the vice president of NVIDIA's automotive business, Wu requires job applicants to have a solid professional background and a strong drive for excellence in technology and products. The large-scale recruitment in China at once indicates NVIDIA's intention to focus on autonomous driving in the country.

In 2024, NVIDIA announced strategic cooperation agreements with four Chinese companies to jointly promote progress in the field of intelligent driving. These four automotive companies are Li Auto, Great Wall Motors, Zeekr, and Xiaomi.

It is reported that in the strategic cooperation with Li Auto, NVIDIA's NVIDIA DRIVE Thor centralized vehicle computing platform was chosen as the intelligent driving system platform for its next generation of vehicles. The application of this platform will help improve Li Auto's intelligent driving capabilities, bringing users a more convenient and safer travel experience. The other three companies have indicated that their new generation of autonomous driving systems will adopt NVIDIA's NVIDIA DRIVE Orin computing platform.

Regardless of how much this is related to Wu Xinzhu, it is clear that NVIDIA's efforts in China have already yielded results. In the race where Apple quietly withdrew, where can Wu Xinzhu lead NVIDIA, a company with a market value of 2 trillion? Both Jen-Hsun Huang and Elon Musk are eager to know the answer.