The Language Layer Does Not Matter Andrej Karpathy does not use C. He does not need to. His work sits at the model research and pedagogy layer where Python is the lingua franca. The real compute happens in C and CUDA underneath, but that layer is invisible to him. Training a model takes ten hours in C++ and ten hours in Python — the GPU is the bottleneck, not the host language. ...
(Image source: https://cs.stanford.edu/people/eroberts/courses/soco/projects/2008-09/tony-hoare/images/hoare%20main.jpg ) In case you like code more than prose about the code, here is the code. That is vibed. Quality is average, bordering on bad. I spent some time, in between other tasks, and made what is I might claim, elegant, human optimised code. ( https://godbolt.org/z/9M8oj34TT ) Those improvements need never stop, of course. The Foundation Everyone Forgot — Almost Before Simula 67. Before Smalltalk. Before anyone had coined the term object-oriented programming. Tony Hoare formalized the concept of the discriminated union in his seminal 1965 paper, “Record Handling.” ...