Jensen Huang

Jensen Huang · 23,765 词 · 查看原文 ↗
AI 与机器学习技术与编程商业与创业音乐与艺术政治与社会
🤖 AI 智能总结

英伟达CEO黄仁勋谈AI芯片、算力与未来计算

这是 Lex Fridman 与英伟达 CEO 黄仁勋的深度对话,涵盖了 NVIDIA 从 GPU 到 AI 计算平台的战略转型、机架级系统设计哲学、AI Scaling Laws 的现状与瓶颈,以及黄仁勋对 AGI 时间线和未来计算架构的独特见解。

NVIDIAAI芯片算力AGI领导力供应链

黄仁勋(Jensen Huang)是英伟达(NVIDIA)联合创始人兼 CEO,将 NVIDIA 从游戏 GPU 公司打造为 AI 时代最重要的计算平台,市值一度超过3万亿美元,被认为是 AI 革命的核心推动者。

📌 核心观点
  • 黄仁勋解释了「极端协同设计」(extreme co-design)的必要性:当 AI 问题规模超出单台计算机时,必须同时优化 GPU、CPU、内存、网络、电源冷却和软件栈,才能实现超线性扩展,而非仅仅堆砌更多计算机。
  • 他管理着超过60名直接汇报者,刻意不做一对一会议,而是通过大型会议让信息在组织内透明流动——他认为公司架构应该反映其所处的环境,而非套用通用的层级模板。
  • 关于 AI Scaling Laws,黄仁勋认为当前最大的瓶颈不是算法,而是供应链(HBM 内存、TSMC 产能)和电力基础设施,这些物理约束比算法挑战更难突破。
  • 他对 NVIDIA 的护城河有清醒认识:不仅是 CUDA 生态,更是整个软件栈(cuDNN、NCCL、TensorRT 等)和极端协同设计能力,这需要数十年积累,竞争对手很难在短期内复制。
  • 黄仁勋预测 AGI 将在本十年内到来,并认为未来的数据中心将像电力公用事业一样成为基础设施,NVIDIA 有望成为价值10万亿美元的公司。
✨ 金句摘录
黄仁勋:我的直接汇报有60多人,我不做一对一会议——因为如果你有60个人汇报,你根本没法做一对一还能完成工作。
黄仁勋:公司的架构应该反映它所处的环境,而不是套用汉堡式或软件式的通用组织图。
黄仁勋:你加了10000台计算机,但你希望它快一百万倍——这就是为什么极端协同设计是必要的。
📋 章节目录
0:00 Introduction · 介绍
0:33 Extreme co-design and rack-scale engineering · 极端协同设计和机架规模工程
3:18 How Jensen runs NVIDIA · Jensen 如何运行 NVIDIA
22:40 AI scaling laws · 人工智能缩放法则
37:40 Biggest blockers to AI scaling laws · 人工智能扩展法则的最大障碍
39:23 Supply chain · 供应链
41:18 Memory · 记忆
47:24 Power · 力量
52:43 Elon and Colossus · 伊隆与巨像
56:11 Jensen’s approach to engineering and leadership · Jensen 的工程和领导方法
1:01:37 China · 中国
1:09:50 TSMC and Taiwan · 台积电和台湾
1:15:04 NVIDIA’s moat · NVIDIA 的护城河
1:20:41 AI data centers in space · 太空人工智能数据中心
1:24:30 Will NVIDIA be worth $10 trillion? · NVIDIA 的市值会达到 10 万亿美元吗?
1:34:39 Leadership under pressure · 压力下的领导力
1:48:25 Video games · 电子游戏
1:55:16 AGI timeline · 通用人工智能时间表
1:57:29 Future of programming · 编程的未来
2:11:01 Consciousness · 意识
🔑 关键词
jensenhuangcompanynvidiadongoingfuturedatacomputingincredibleeverybodyscaledoingcudaengineeringtechnologydesignimportantputjob
💬 精彩语录
"Okay? And so it’s been a long time since computer vision has been superhuman. And so the prediction was radiologists would go away because studying radiology scans was a thing of the past. AI will do that. Well, they were absolutely right. Computer vision is completely superhuman. Every radiology platform and package today is driven by AI, and yet the number of radiologists grew. And so the question is why? And we now have a shortage of radiologists in the world. And so, one, the alarmist warning went too far and it scared people from doing this profession that is so important to society. And so it did harm. Now, why was it wrong? The reason why is because the purpose of a radiologist, the purpose is to diagnose disease and help patients and doctors diagnose disease."
好的?因此,计算机视觉成为超人类已经有很长一段时间了。因此,预测放射科医生将会离开,因为研究放射扫描已成为过去。人工智能会做到这一点。嗯,他们是绝对正确的。计算机视觉完全是超人的。如今,每个放射学平台和软件包都是由人工智能驱动的,但放射科医生的数量却在增加。所以问题是为什么?现在世界上放射科医生短缺。因此,第一,危言耸听的警告太过分了,它吓坏了人们从事这个对社会如此重要的职业。所以它确实有害。现在,为什么错了?之所以是因为放射科医生的目的,目的就是诊断疾病,帮助患者和医生诊断疾病。
— Jensen Huang (01:59:39)
"I don’t know if the chip will ever get nervous. And that’s the, you know, of course, the conditions by which that causes anxiety or nervousness or whatever emotion. I believe that AI will be able to recognize those and understand those. I don’t think my chips will feel those. And therefore, the… How that anxiety, how that feeling, how that excitement, how that, how that, you know… All of those feelings manifest in human performance. For example, extremely amazing human performance, athletic performance, you know, average or lesser than average. That entire spectrum of human performance that comes out of exactly the same circumstances for different people, manifesting a different outcome, manifesting a different performance."
我不知道芯片是否会紧张。当然,这就是导致焦虑、紧张或其他情绪的条件。我相信人工智能将能够识别并理解这些。我不认为我的芯片会感觉到这些。因此,……那种焦虑、那种感觉、那种兴奋、那种、那种感觉,你知道……所有这些感觉都体现在人类的表现中。例如,极其惊人的人类表现,运动表现,你知道,平均或低于平均水平。对于不同的人来说,在完全相同的环境下产生的整个人类表现,表现出不同的结果,表现出不同的表现。
— Jensen Huang (02:11:18)
"And it’s a system that… You know, it’s something that we do that includes perception and understanding and reasoning and the ability to do plan. And, you know, that loop, that loop, is the… Fundamentally what intelligence is. Intelligence is not one word that is exactly equal to humanity. And that’s, I think it’s really important to separate the two. We have two words for that. I’m not… I don’t over-fantasize about, and I don’t over-romanticize about intelligence. Intelligence is… And people have heard me say it before, I actually think intelligence is a commodity. I’m surrounded by intelligent people. And I’m surrounded by intelligent people more intelligent than I am in each one of the spaces that they’re in."
这是一个系统……你知道,这是我们所做的事情,包括感知、理解、推理以及制定计划的能力。而且,你知道,那个循环,那个循环,就是……从根本上来说,智力是什么。智能这个词并不完全等同于人类。也就是说,我认为将两者分开非常重要。我们有两个词来形容。我不是……我不会对智力过度幻想,也不会过度浪漫化智力。智力是……人们以前听我说过,我实际上认为智力是一种商品。我周围都是聪明人。我周围都是聪明人,他们所在的每一个空间都比我更聪明。
— Jensen Huang (02:13:46)
"Yeah, it turned out to have been a good decision. I think the… So, here’s the way it went. So, we invented this thing called CUDA, and it expanded the aperture of applications that we can accelerate with our accelerator. The question is, how do we attract developers to CUDA? Because a computing platform is all about developers. And developers don’t come to a computing platform just because, you know, it could perform something interesting. They come to a computing platform because the install base is large. Because a developer, like anybody else, wants to develop software that reaches a lot of people. So, the install base is, in fact, the single most important part of an architecture. The architecture could attract enormous amounts of criticism."
是的,事实证明这是一个很好的决定。我认为……所以,事情是这样的。因此,我们发明了这种称为 CUDA 的东西,它扩大了我们可以使用加速器加速的应用程序的范围。问题是,我们如何吸引开发者使用 CUDA?因为计算平台的核心就是开发人员。开发人员来到计算平台并不仅仅是因为它可以执行一些有趣的事情。他们之所以选择计算平台,是因为安装基数很大。因为开发人员和其他人一样,希望开发出能够惠及很多人的软件。因此,安装基础实际上是架构中最重要的部分。该架构可能会招致大量批评。
— Jensen Huang (00:11:27)
"Well, first of all, I’m informed by a lot of curiosity. At some point, there’s a reasoning system that convinces me so clearly this outcome will happen. That this will happen. And so I believe it in my mind, and when I believe it in my mind, you know how it is. You manifest a future and that future is so convincing, there’s no way it won’t happen. There’s a lot of suffering in between, but you’ve gotta believe what you believe."
嗯,首先,我充满好奇心。在某些时候,有一个推理系统让我清楚地相信这个结果将会发生。这将会发生。所以我心里相信它,当我心里相信它时,你就知道它是怎么回事。你展现了一个未来,而这个未来是如此令人信服,它不可能不会发生。中间有很多痛苦,但你必须相信你所相信的。
— Jensen Huang (00:17:14)
🎙️ 完整对话(399 条)
Lex Fridman (00:00:00)
The following is a conversation with Jensen Huang, CEO of NVIDIA, one of the most important and influential companies in the history of human civilization. NVIDIA is the engine powering the AI revolution, and a lot of its success can be directly attributed to Jensen’s sheer force of will and his many brilliant bets and decisions as a leader, engineer, and innovator. This is Lex Fridman Podcast. And now dear friends, here’s Jensen Huang. Extreme co-design and rack-scale engineering
以下是与人类文明史上最重要、最具影响力的公司之一英伟达首席执行官黄仁勋的对话。 NVIDIA 是人工智能革命的引擎,它的成功在很大程度上可以直接归功于 Jensen 的坚定意志力以及他作为领导者、工程师和创新者的许多出色的赌注和决策。这是莱克斯·弗里德曼播客。一个
Lex Fridman (00:00:33)
You’ve propelled NVIDIA into a new era in AI, moving beyond his focus on chip scale design to now rack scale design.
你们推动 NVIDIA 进入了 AI 的新时代,从专注于芯片级设计转向了现在的机架级设计。
Lex Fridman (00:00:42)
And I think it’s fair to say that winning for NVIDIA for a long time used to be about building the best GPU possible, and you still do, but now you’ve expanded that to extreme co-design of GPU, CPU memory, networking, storage, power cooling, software, the rack itself, the pod that you’ve announced, and even the data center. So let’s talk about extreme co-design. What is the hardest part of co-designing a system with that many complex components and design variables?
我认为可以公平地说,长期以来 NVIDIA 的胜利在于构建尽可能最好的 GPU,现在仍然如此,但现在您已将其扩展到 GPU、CPU 内存、网络、存储、电源冷却、软件、机架本身、您宣布的 Pod 甚至数据中心的极端协同设计。那么我们来谈谈极限协同设计。协同设计最难的部分是什么
Lex Fridman (00:01:11)
Yeah, thanks for that question. So first of all, the reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU. The problem that you’re trying to solve is you would like to go faster than the number of computers that you add. So you added 10,000 computers, but you would like it to go a million times faster. Then all of a sudden you have to take the algorithm, you have to break up the algorithm, you have to refactor it, you have to shard the pipeline, you have to shard the data, you have to shard the model. Now all of a sudden when you distribute the problem this way, not just scaling up the problem, but you’re distributing the problem, then everything gets in the way.
是的,谢谢你提出这个问题。首先,极端协同设计之所以必要,是因为这个问题不再适合在一台计算机内由一个 GPU 加速。您试图解决的问题是您希望运行速度比您添加的计算机数量更快。因此,您添加了 10,000 台计算机,但您希望它的运行速度快一百万倍。然后突然之间
Lex Fridman (00:02:03)
This is the Amdahl’s law problem where the amount of speed up you have for something depends on how much of the total workload it is. And so if computation represents 50% of the problem, and I sped up computation infinitely like a million times, you know, I only sped up the total workload by a factor of two. Now all of a sudden, not only do you have to distribute a computation, you have to shard the pipeline somehow. You also have to solve the networking problem because you’ve got all of these computers are all connected together. And so distributed computing at the scale that we do, the CPU is a problem, the GPU is a problem, the networking is a problem, the switching is a problem. And distributing the workload across all these computers is a problem.
这是阿姆达尔定律问题,其中某件事的加速量取决于它占总工作量的多少。因此,如果计算占问题的 50%,并且我将计算速度无限加快了一百万倍,你知道,我只将总工作负载速度加快了两倍。现在突然之间,您不仅必须分配计算,还必须对 t 进行分片
Jensen Huang (00:02:57)
It’s just a massively complex computer science problem. And so we just gotta bring every technology to bear. Otherwise, we scale up linearly or we scale up based on the capabilities of Moore’s Law, which has largely slowed because Dennard scaling has slowed. How Jensen runs NVIDIA
这只是一个非常复杂的计算机科学问题。因此,我们必须运用所有技术。否则,我们会线性扩大规模,或者根据摩尔定律的能力扩大规模,摩尔定律的速度在很大程度上已经放缓,因为登纳德缩放速度已经放缓。 Jensen 如何运行 NVIDIA
Lex Fridman (00:03:16)
I’m sure there’s trade-offs there. Plus you have a complete disparate disciplines here. I’m sure you have specialists in each one of these high bandwidth memory, the network and the NVLink, the NICs, the optics and the copper that you’re doing, the power delivery, the cooling, all of that. I mean, there’s like world experts in each of those. How do you get ’em in a room together to figure out-
我确信这其中存在权衡。另外,这里有完全不同的学科。我确信你们在这些高带宽内存、网络和 NVLink、NIC、光学和铜缆、电力传输、冷却等方面都有专家。我的意思是,每一个领域都有世界专家。你如何让他们一起在一个房间里弄清楚-
Jensen Huang (00:03:34)
That’s why my staff is so large. Yeah.
这就是为什么我的员工如此庞大。是的。
Lex Fridman (00:03:37)
What’s the pro- can you take me through the process of the specialists and the generalists? Like how do you put together the rack when you know the s- the set of things you have to shove into a rack together? Like what does that process look like of designing it all together?
有什么问题-你能带我了解一下专家和通才的过程吗?就像当您知道必须将一组东西一起推入架子时,如何将架子组装在一起?一起设计的过程是什么样的?
Lex Fridman (00:03:51)
Yeah. There’s the first question, which is: what is extreme co-design? We’re optimizing across the entire stack of software from architectures to chips, to systems, to system software, to the algorithms, to the applications. That’s one layer. The second thing that you and I just talked about goes beyond CPUs and GPUs and networking chips and scale up switches and scale out switches. And then of course, you gotta include power and cooling and all of that because all these computers are extremely power hungry. They do a lot of work and they’re very energy efficient, but they in aggregate still consume a lot of power. And so that’s one. The first question is, what is it?
是的。第一个问题是:什么是极限协同设计?我们正在优化整个软件堆栈,从架构到芯片、系统、系统软件、算法和应用程序。这是一层。你和我刚才谈到的第二件事超出了 CPU、GPU 和网络芯片以及纵向扩展交换机和横向扩展交换机的范围。然后是co
Lex Fridman (00:04:34)
The second question is, why is it, and we just spoke about the reason, you know you want to distribute the workload so that you can exceed the benefit of just increasing the number of computers. And the, and then the third question is, how is it, how do you do it?
第二个问题是,为什么会这样,我们刚才谈到了原因,你知道你想要分配工作负载,这样你就可以超越仅仅增加计算机数量所带来的好处。然后第三个问题是,怎么样,你是怎么做到的?
Jensen Huang (00:04:51)
And, and that’s the, that’s kind of the miracle of this company. You know, when you’re designing a computer, you have to have an operating system of computers. When you’re designing a company, you should first think about what is it that you want the company to produce. You know, I see a lot of companies’ organization charts, and they all look the same. Hamburger organization charts, soft organization charts, and car company organization charts. They all look the same. And it doesn’t make any sense to me. You know, the goal of a comp- of a company is to be the machinery, the mechanism, the system that produces the output. And that output is the product that we like to create. It is also designed, the architecture of the company should reflect the environment by which it exists.
这就是这家公司的奇迹。你知道,当你设计一台计算机时,你必须有一个计算机操作系统。当你设计一家公司时,你应该首先考虑你希望公司生产什么。你知道,我看过很多公司的组织结构图,它们看起来都一样。汉堡组织结构图,软组织
Jensen Huang (00:05:36)
It almost directly says what you should do with the organization. My direct staff is 60 people. You know, I don’t have one-on-ones with ’em because it’s impossible. You can’t have 60 people on your staff if you’re, you know, gonna get work done and-
它几乎直接说明了你应该对组织做什么。我的直接员工有 60 人。你知道,我不会与他们进行一对一的交流,因为这是不可能的。如果你要完成工作,你就不可能拥有 60 名员工,而且——
Lex Fridman (00:05:51)
So you still have 60 reports. You still have across-
所以您还有 60 份报告。你还有跨-
Lex Fridman (00:05:53)
More, yeah.
更多,是的。
Lex Fridman (00:05:54)
More. And most stars at least have a foot in engineering.
更多的。大多数明星至少都涉足工程学领域。
Jensen Huang (00:05:59)
Almost all of them. There’s experts in memory, there’s experts in CPUs, there’s experts in optical. All-
几乎全部。有内存专家、CPU 专家、光学专家。全部-
Lex Fridman (00:06:06)
That’s incredible.
那真是难以置信。
Lex Fridman (00:06:06)
Yeah, GPUs and- Architecture, algorithms, design-
是的,GPU 和——架构、算法、设计——
Lex Fridman (00:06:11)
So, you constantly have an eye on the entire stack, and you’re having to have, like, intense discussions about the design of the entire stack?
那么,您需要不断关注整个堆栈,并且必须对整个堆栈的设计进行激烈的讨论?
Lex Fridman (00:06:18)
And no conversation is ever one person. That’s why I don’t do one-on-ones. We present a problem and all of us attack it. You know, because we’re doing extreme co-design. And literally, the company is doing extreme co-design all the time.
Lex Fridman (00:06:33)
So, even if you’re talking about a particular component, like cooling, networking, everybody’s listening in?
Lex Fridman (00:06:40)
Yeah, exactly.
Lex Fridman (00:06:41)
And they can contribute, “Well, this doesn’t work for the power distribution. This doesn’t-“
Lex Fridman (00:06:45)
Exactly.
Lex Fridman (00:06:45)
“… This doesn’t work for the memory. This doesn’t work for this.”
Jensen Huang (00:06:49)
Exactly. And whoever wants to tune out, tune out. You know what I’m saying? And the reason for that is because the people who are on the staff, they know when to pay attention. There’s supposed… You know, it’s something they could have contributed to, they didn’t contribute to, “I’m going to call them out.” You know? And so, “Hey, come on, let’s get in here.”
Lex Fridman (00:07:07)
So, as you mentioned, NVIDIA is this company that’s adapting to the environment. So, which point can you say, did the environment change and began adapting sort of secretly- … in the early days from GPU for gaming, maybe the early deep learning revolution to we’re now going to start thinking of it as an AI factory? What does NVIDIA do? It produces AI; let’s build a factory that makes AI.
Jensen Huang (00:07:32)
I could reason through that systematically. We started out as an accelerator company. But the problem with accelerators is that the application domain’s too narrow. It has the benefit of being incredibly optimized for the job. You know, any specialist has that benefit. The problem with intense specialization is that, of course, your market reach is narrower, but that’s even fine. The problem is, the market size also dictates your R&D capacity. And your R&D capacity ultimately dictates the influence and impact that you can possibly have in computing. And so, when we first started out as an accelerator, very specific accelerator, we always knew that was going to be our first step.
Jensen Huang (00:08:23)
We had to find a way to become accelerated computing. But the problem is, when you become a computing company, it’s too general purpose and it takes away from your specialization. The tur- I connected two words that actually have fundamental tension. The better computing company we become, the worse we became as a specialist. The more of a specialist, the less capacity we have to do overall computing. And so, that… And I connected those two words together on purpose, that the company has to find that really narrow path, step by step by step, to expand our aperture of computing, but not give up on the most important specialization that we had. Okay, so the first step that we took beyond acceleration was we invented a programmable pixel shader.
查看原始文字稿 ↗
🔗 相关节目