Introduction
We’re very excited about the positive reception of Mojo since its launch as well as the community of people building around it. Given new Large Language Model (LLM) powered developer tools like Copilot and Ghostwriter, many developers are wondering about the future of programming – do programming languages still matter when AI writes the code?
This is a great question! It cuts to the heart of developer workflows, allows us to reflect on the core purpose of programming tools more broadly, and encourages us to share our perspective on where coding technologies are going over the long term. First, let’s explore what a programming language is for across three critical dimensions.
The “Human to computer” specification
One of my early passions when learning how to code was understanding how the whole stack worked, and I got involved in programming language design and implementation as a student. As part of that, my first viewpoint was that a programming language is an abstraction a human uses to express intent to the computer about what a piece of code is supposed to do. I learned that source code is a “recipe” that a compiler or interpreter translates into something the computer can understand.
"The psychological profiling [of a programmer] is mostly the ability to shift levels of abstraction, from low level to high level. To see something in the small and to see something in the large." – Donald Knuth
With this viewpoint, it makes sense for a programming language to be focused on exposing the capabilities of the machine in an accessible way that many humans can understand. A language should be designed to be written with an unambiguous interpretation that allows precise specification of an algorithm, or design, and being concise allows smart programmers to get much done quickly. This is one of the reasons that some “big” languages targeted to expert developers (like C++) accumulate syntactic sugar and core language features that enable the efficient expression of important things.
While this viewpoint is valid, as I gained more experience, I realized that this is only one part of what programming languages do.
The “Human to human” specification
I learned very quickly that most interesting projects are built by teams of people and grow to the size where it is hard to fit all the code in your head. When this happens, new dynamics enter the software development realm – design discussions, code reviews, and 3rd-party library/package integrations. The best and most fulfilling software developments I have ever participated in have always been with groups of talented and dedicated people.
In this context, one quickly realizes that the purpose of a programming language grows to be an abstraction for a human to express intent to another human about the behavior of a program. This still requires unambiguous specification but also shifts the goalposts – a language should be designed to be read, not just written by other humans. Computers are very tolerant and understanding (particularly with the rise of LLMs), so many of us benefit from clear design patterns and easy-to-understand code. Most code ends up being written once, but read and iterated on many times by many people.
"Programs are meant to be read by humans and only incidentally for computers to execute." – Harold Abelson and Gerald Jay Sussman
The consequence is that overly clever syntactic sugar actually starts to cut against the core goals of the language. Specialized syntax and infrequently used features can make it difficult to understand for those who didn’t write it. While LLMs and other tools can help decode or explain overly complicated code, keeping a single, readable source of truth is ideal.
The “Computer to Human” specification
With this lens, LLM-based code generation tools act like a new team member contributing, reading, and manipulating code in a project. There are many different examples with varied features and capabilities, e.g., a human prompt to generate code, an AI expert that reviews your code and offers suggestions to improve it, tools to automate the generation of unit tests, and other new capabilities are coming out all the time. The defining feature of these tools is that the computer generates source code that gets integrated into a product.
While these capabilities are incredible, at least in the near future, these code-generation tools don’t displace the existing functions of a programming language. Indeed, one of the major concerns about language models today is trust – they can give strikingly amazing results in some cases but are often subtly wrong in others, and some are non-deterministic. As such, it becomes even more critical to design a language to be read, not just written by other humans, so we humans can review and approve generated code.
Let’s consider some examples: if you prompted an LLM to build you a mobile app that processes online purchases, would you ship that without reviewing the source code to make sure it does billing right? Or, more extreme yet, would you want to send humans to the moon with code written by an LLM? This begs the ultimate question for developers – what margin of error and cost are we willing to accept? The unreliability of LLMs today means that, as code owners, we need to know if the prompt generated something with the right behavior - what does the generated code actually do? This is also why an LLM that directly generates low-level machine code isn’t interesting for general use cases – few people want to read, review, and validate machine code.
As we look towards the future, we hope that LLMs will augment the developer experience and become more reliable and trustworthy over time. But even if this proves true, LLMs still won’t replace the need for programming languages. LLMs are likely to become a critical extension of the highly productive (“lazy”) developer – a substantial level-up vs copy/pasting from online references. Further, while LLMs will very likely automate away the boilerplate and repetitive parts of programming, there will always be use cases that require the human touch.
While no one knows the future, we think humans will need to be in the loop for quite some time for many applications – particularly where the error margins are low, and resulting costs are high.
What's the ideal programming language for LLMs to output?
Those deep in software development find themselves surrounded by a zoo of different languages that aim to solve problems in various niches. You might have encountered Python for AI and data science, C and C++ for low-level programming, Javascript or Typescript for the web, Swift and Kotlin when building a mobile app, and CUDA for accelerator programming. These are all valuable languages, but given that LLMs reduce the need to care about how writable a syntax is – what qualities of a programming language matter in this new age?
We believe there are three fundamental aspects of a programming language that would make it particularly useful as we head towards an AI-assisted world – its usability and scalability to many domains, the amount of training data that exists, and a rich and vibrant ecosystem. Let’s take each in turn:
- The first most critical part of a language is the usability and scalability of the language implementation. The best language for an LLM is one that is highly usable and easy to read for humans, but whose implementation can scale to many different use cases and applications. Unfortunately, many language implementations include design decisions that preclude certain applications. For example, mark/sweep garbage collection isn’t ideal for low-level system software and accelerator programming, Python and other interpreted languages aren’t ideal when performance, parallelism, and threading are required, and JVM or .NET-based languages aren’t ideal for use cases that need small and low-dependence binaries.
- To train an LLM that is capable of producing high-quality programs across many different use cases and applications, we need an expansive corpus of training data that seeds the model. An LLM will work much better on a popular and established language like Python which has a large and diverse set of open examples, than a niche or novel language that has no existing code to train on.
- Lastly, we believe that a LLM needs a rich and vibrant ecosystem surrounding it. Even for existing LLM-based solutions, rich communities have already developed prompting libraries, tooling, and expertise enabling next-generation ecosystems to form. With this viewpoint, a language should be designed to unlock a massive community of developers – however we choose to define a developer in this new world, from traditional programming to instruction prompting and beyond.
When we look at the vast number of existing programming languages, we see many points in the space, but they all provide tradeoffs optimized for different niches. How can we move the state-of-the-art forward? In our view, Mojo is a powerful contender to grow into an ideal language for LLMs, as it meets all three fundamental aspects above.
Our approach with Mojo
We’ve learned a lot from building other compiler and programming language systems (e.g., Clang/C++, Swift, etc) over the last 20+ years. From that experience, we are building Mojo to:
- Be a fully compatible superset of Python, benefiting from its easy to read and understandable syntax and enabling its large community of developers to already know how to write Mojo!
- Support system programming features and hardware accelerators that extend the performance and reach of Python into new domains as we move into a new parallel-computing world.
- Be fully integrated with the existing Python ecosystem, extending and benefiting from all of the existing packages. We will also build seamless C and C++ interoperability to lift (and benefit from) work in those communities over time.
- Provide a new high-performance heterogeneous compiler and runtime implementation that benefits from state-of-the-art techniques.
As a consequence, we believe Mojo fits the perfect sweet spot for LLMs to generate and output highly scalable code, because it combines the human readability and usability of Python, but extends it with powerful lower-level systems features that enable it to scale across more hardware and drive the next set of the world’s applications and use cases.
We think LLMs will continue to unlock creativity and productivity across many languages, but we also believe Mojo will be well prepared to lift collaborative software development to the next level and bring programming into new frontiers.
Mojo is still early – we unveiled its 0.1 release last month – but we are in this for the long term and have clear goals. If you are interested in being a part of Mojo early on, please join our community or apply to help us build it!
