Building a backend from scratch using only OpenAI Codex
In the last few weeks, I and the rest of our team have been playing around a lot with OpenAI's Codex Model, as we've incorporated it into our latest feature (the suggester), and we've been generally impressed by it's performance.
"OpenAI Codex is an artificial intelligence model developed by OpenAI. It parses natural language and generates code in response. It is used to power GitHub Copilot, a programming autocompletion tool developed for Visual Studio Code. Codex is a descendant of OpenAI's GPT-3 model, fine-tuned for use in programming applications."
I had the idea of a challenge for myself: Create something useful, from scratch, using only Codex.
My goal was to create a backend for a TodoMVC-style app. In our case, it will be a backend for a tasklist with tasks that can be added, listed, and marked as completed. The frontend is to be built manually (in SvelteKit).
First, I tried to create the backend in Go, but Codex continuously messed up the syntax when inserting new code, and usually ended up adding a new function withing another one (syntax error!)
I switched strategy, and attempted to generate a Python app instead, and it worked great. The continuation of this blog post is a series of prompts used, together with comments about what worked and what didn't end up working out.
I ended up using the "Codex" in the "edit" mode as it makes the prompt design easier. The edit mode takes two inputs, a file and a instruction/prompt, and spits out a new version of the input file. For the input file, I started with an empty file, and for later prompts the output of the previous step was used.
Here we go! First prompt:
Create a hello world flask webserver
Let's get started! In my first attempt I used "python" instead of "flask" in the prompt. The code worked, but was too low level, and I could sense that it would get complicated too fast.
So I changed the prompt to "flask", and off we went! I'm no Flask expert, but this looks good to me.
Codex does not help me install dependencies, but I was able to get started with
pip3 install flask.
If you're interested, the server is up and running here.
Add a /tasks endpoint listing all tasks in a sqlite database
A Hello World might be cool and all, but it's time to write some real code.
Codex adds the endpoint correctly, LGTM! The file or prompt has no previous mention of sqlite or table names, so Codex makes something up for us, so far so good!
before_first_request: Make sure that the tasks table exists (id auto incremental id, description text, done bool)
The tasks.db database does not exist yet, and has no tables.
Let's create a table structure, and ask Codex to set it up if it does not exist.
This took some tweaking to get right: prompts like "
setup ... before the app starts" writes code, but it does not work and is not hooked up to Flask in any way.
Adding the keyword
before_first_request fixes it and Codex does the right thing!
Update the tasks() method to return tasks as objects
Let's make the /tasks endpoint easier to consume (it's currently returning a list of lists), return as a list of objects.
Add a /add endpoint that inserts new entries to the tasks table.
There is no CRUD without the C!
Looks good, but beware: sometimes Codex writes code vulnerable to SQL injection. When that happens tough, I was able to prevent it by adding "safely" to the prompt.
Add a /update endpoint that updates task descriptions and done status
Codex is on a streak, this works nicely!
I think that we have everything we need to run our app now, let's deploy it!
Add CORS headers to allow connections from any host
Uuuuh! Our friend CORS showed up and stopped us from enjoying getting everything on the first try. We need to add a CORS handler.
After this, the app was up and running with no problems!
Update the add() method to return the id of the task created in json
The response from /add was a bit simple, and to improve the frontend (that I developer manually at this stage), it was nice to have the ID of the task in the response.
This step is just for fun, and with a simple "Format" prompt, the code was formatted to be compliant with PEP 8!
Brb while I run:
pip uninstall black && pip install codex-formatter
before_first_request, before conn.close: if the tasks table is empty, add three rows
To make [the demo] nicer to look at, let's add some example data!
I can sense that Codex is getting annoyed with me, and asks me to learn Python by myself! 😬
That's it, our backend is done! You can check it out, and use the frontend at todo.codeball.ai.
Developing with Codex is a bit special, and it sometimes takes a few attempts to get it to write exactly what you want it to. But in broad strokes, getting from nothing to something in just 10 prompts is really impressive and encouraging.
Codex continues to amaze me, and I'm sure that this task could be done in fewer steps (maybe even one?) with some prompt engineering. I'd like to start a prompt golf challenge! If you have access to Codex, can you re-create this backend in fewer steps? As few characters as possible?
Would be cool to see how far we can take this, looking forward to hear in the comments on Hacker News.