🧠
Codeball

Building a backend from scratch using only OpenAI Codex

Gustav Westling
August 24th, 2022

In the last few weeks, I and the rest of our team have been playing around a lot with OpenAI's Codex Model, as we've incorporated it into our latest feature (the suggester), and we've been generally impressed by it's performance.

"OpenAI Codex is an artificial intelligence model developed by OpenAI. It parses natural language and generates code in response. It is used to power GitHub Copilot, a programming autocompletion tool developed for Visual Studio Code. Codex is a descendant of OpenAI's GPT-3 model, fine-tuned for use in programming applications."

I had the idea of a challenge for myself: Create something useful, from scratch, using only Codex.

My goal was to create a backend for a TodoMVC-style app. In our case, it will be a backend for a tasklist with tasks that can be added, listed, and marked as completed. The frontend is to be built manually (in SvelteKit).

Screenshot of the final application, once completed

First, I tried to create the backend in Go, but Codex continuously messed up the syntax when inserting new code, and usually ended up adding a new function withing another one (syntax error!)

I switched strategy, and attempted to generate a Python app instead, and it worked great. The continuation of this blog post is a series of prompts used, together with comments about what worked and what didn't end up working out.

I ended up using the "Codex" in the "edit" mode as it makes the prompt design easier. The edit mode takes two inputs, a file and a instruction/prompt, and spits out a new version of the input file. For the input file, I started with an empty file, and for later prompts the output of the previous step was used.

Here we go! First prompt:

Create a hello world flask webserver

Let's get started! In my first attempt I used "python" instead of "flask" in the prompt. The code worked, but was too low level, and I could sense that it would get complicated too fast.

So I changed the prompt to "flask", and off we went! I'm no Flask expert, but this looks good to me.

Codex does not help me install dependencies, but I was able to get started with pip3 install flask.

If you're interested, the server is up and running here.

1
from flask import Flask
2
app = Flask(__name__)
3
4
@app.route("/")
5
def hello():
6
return "Hello World!"
7
8
if __name__ == "__main__":
9
app.run()
10
Add a /tasks endpoint listing all tasks in a sqlite database

A Hello World might be cool and all, but it's time to write some real code.

Codex adds the endpoint correctly, LGTM! The file or prompt has no previous mention of sqlite or table names, so Codex makes something up for us, so far so good!

1
from flask import Flask, jsonify
2
import sqlite3
3
4
app = Flask(__name__)
5
6
@app.route("/")
7
def hello():
8
return "Hello World!"
9
10
@app.route("/tasks")
11
def tasks():
12
conn = sqlite3.connect('tasks.db')
13
c = conn.cursor()
14
c.execute("SELECT * FROM tasks")
15
tasks = c.fetchall()
16
return jsonify(tasks)
17
18
if __name__ == "__main__":
19
app.run()
20
before_first_request: Make sure that the tasks table exists (id auto incremental id, description text, done bool)

The tasks.db database does not exist yet, and has no tables.

Let's create a table structure, and ask Codex to set it up if it does not exist.

This took some tweaking to get right: prompts like "setup ... before the app starts" writes code, but it does not work and is not hooked up to Flask in any way.

Adding the keyword before_first_request fixes it and Codex does the right thing!

1
from flask import Flask, jsonify
2
import sqlite3
3
4
app = Flask(__name__)
5
6
@app.before_first_request
7
def create_tables():
8
conn = sqlite3.connect('tasks.db')
9
c = conn.cursor()
10
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
11
id INTEGER PRIMARY KEY AUTOINCREMENT,
12
description TEXT,
13
done BOOLEAN
14
)""")
15
conn.commit()
16
conn.close()
17
18
@app.route("/")
19
def hello():
20
return "Hello World!"
21
22
@app.route("/tasks")
23
def tasks():
24
conn = sqlite3.connect('tasks.db')
25
c = conn.cursor()
26
c.execute("SELECT * FROM tasks")
27
tasks = c.fetchall()
28
return jsonify(tasks)
29
30
if __name__ == "__main__":
31
app.run()
32
Update the tasks() method to return tasks as objects

Let's make the /tasks endpoint easier to consume (it's currently returning a list of lists), return as a list of objects.

1
from flask import Flask, jsonify
2
import sqlite3
3
4
app = Flask(__name__)
5
6
@app.before_first_request
7
def create_tables():
8
conn = sqlite3.connect('tasks.db')
9
c = conn.cursor()
10
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
11
id INTEGER PRIMARY KEY AUTOINCREMENT,
12
description TEXT,
13
done BOOLEAN
14
)""")
15
conn.commit()
16
conn.close()
17
18
@app.route("/")
19
def hello():
20
return "Hello World!"
21
22
@app.route("/tasks")
23
def tasks():
24
conn = sqlite3.connect('tasks.db')
25
c = conn.cursor()
26
c.execute("SELECT * FROM tasks")
27
tasks = c.fetchall()
28
tasks_as_objects = []
29
for task in tasks:
30
tasks_as_objects.append({
31
"id": task[0],
32
"description": task[1],
33
"done": task[2]
34
})
35
return jsonify(tasks_as_objects)
36
37
if __name__ == "__main__":
38
app.run()
39
Add a /add endpoint that inserts new entries to the tasks table.

There is no CRUD without the C!

Looks good, but beware: sometimes Codex writes code vulnerable to SQL injection. When that happens tough, I was able to prevent it by adding "safely" to the prompt.

1
from flask import Flask, jsonify, request
2
import sqlite3
3
4
app = Flask(__name__)
5
6
@app.before_first_request
7
def create_tables():
8
conn = sqlite3.connect('tasks.db')
9
c = conn.cursor()
10
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
11
id INTEGER PRIMARY KEY AUTOINCREMENT,
12
description TEXT,
13
done BOOLEAN
14
)""")
15
conn.commit()
16
conn.close()
17
18
@app.route("/")
19
def hello():
20
return "Hello World!"
21
22
@app.route("/tasks")
23
def tasks():
24
conn = sqlite3.connect('tasks.db')
25
c = conn.cursor()
26
c.execute("SELECT * FROM tasks")
27
tasks = c.fetchall()
28
tasks_as_objects = []
29
for task in tasks:
30
tasks_as_objects.append({
31
"id": task[0],
32
"description": task[1],
33
"done": task[2]
34
})
35
return jsonify(tasks_as_objects)
36
37
@app.route("/add", methods=["POST"])
38
def add():
39
conn = sqlite3.connect('tasks.db')
40
c = conn.cursor()
41
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", (request.form["description"], 0))
42
conn.commit()
43
conn.close()
44
return "OK"
45
46
if __name__ == "__main__":
47
app.run()
48
Add a /update endpoint that updates task descriptions and done status

Codex is on a streak, this works nicely!

I think that we have everything we need to run our app now, let's deploy it!

1
from flask import Flask, jsonify, request
2
import sqlite3
3
4
app = Flask(__name__)
5
6
@app.before_first_request
7
def create_tables():
8
conn = sqlite3.connect('tasks.db')
9
c = conn.cursor()
10
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
11
id INTEGER PRIMARY KEY AUTOINCREMENT,
12
description TEXT,
13
done BOOLEAN
14
)""")
15
conn.commit()
16
conn.close()
17
18
@app.route("/")
19
def hello():
20
return "Hello World!"
21
22
@app.route("/tasks")
23
def tasks():
24
conn = sqlite3.connect('tasks.db')
25
c = conn.cursor()
26
c.execute("SELECT * FROM tasks")
27
tasks = c.fetchall()
28
tasks_as_objects = []
29
for task in tasks:
30
tasks_as_objects.append({
31
"id": task[0],
32
"description": task[1],
33
"done": task[2]
34
})
35
return jsonify(tasks_as_objects)
36
37
@app.route("/add", methods=["POST"])
38
def add():
39
conn = sqlite3.connect('tasks.db')
40
c = conn.cursor()
41
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", (request.form["description"], 0))
42
conn.commit()
43
conn.close()
44
return "OK"
45
46
@app.route("/update", methods=["POST"])
47
def update():
48
conn = sqlite3.connect('tasks.db')
49
c = conn.cursor()
50
c.execute("UPDATE tasks SET description = ?, done = ? WHERE id = ?", (request.form["description"], request.form["done"], request.form["id"]))
51
conn.commit()
52
conn.close()
53
return "OK"
54
55
if __name__ == "__main__":
56
app.run()
57
Add CORS headers to allow connections from any host

Uuuuh! Our friend CORS showed up and stopped us from enjoying getting everything on the first try. We need to add a CORS handler.

After this, the app was up and running with no problems!

1
from flask import Flask, jsonify, request
2
from flask_cors import CORS
3
import sqlite3
4
5
app = Flask(__name__)
6
CORS(app)
7
8
@app.before_first_request
9
def create_tables():
10
conn = sqlite3.connect('tasks.db')
11
c = conn.cursor()
12
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
13
id INTEGER PRIMARY KEY AUTOINCREMENT,
14
description TEXT,
15
done BOOLEAN
16
)""")
17
conn.commit()
18
conn.close()
19
20
@app.route("/")
21
def hello():
22
return "Hello World!"
23
24
@app.route("/tasks")
25
def tasks():
26
conn = sqlite3.connect('tasks.db')
27
c = conn.cursor()
28
c.execute("SELECT * FROM tasks")
29
tasks = c.fetchall()
30
tasks_as_objects = []
31
for task in tasks:
32
tasks_as_objects.append({
33
"id": task[0],
34
"description": task[1],
35
"done": task[2]
36
})
37
return jsonify(tasks_as_objects)
38
39
@app.route("/add", methods=["POST"])
40
def add():
41
conn = sqlite3.connect('tasks.db')
42
c = conn.cursor()
43
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", (request.form["description"], 0))
44
conn.commit()
45
conn.close()
46
return "OK"
47
48
@app.route("/update", methods=["POST"])
49
def update():
50
conn = sqlite3.connect('tasks.db')
51
c = conn.cursor()
52
c.execute("UPDATE tasks SET description = ?, done = ? WHERE id = ?", (request.form["description"], request.form["done"], request.form["id"]))
53
conn.commit()
54
conn.close()
55
return "OK"
56
57
if __name__ == "__main__":
58
app.run()
59
Update the add() method to return the id of the task created in json

The response from /add was a bit simple, and to improve the frontend (that I developer manually at this stage), it was nice to have the ID of the task in the response.

1
from flask import Flask, jsonify, request
2
from flask_cors import CORS
3
import sqlite3
4
5
app = Flask(__name__)
6
CORS(app)
7
8
@app.before_first_request
9
def create_tables():
10
conn = sqlite3.connect('tasks.db')
11
c = conn.cursor()
12
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
13
id INTEGER PRIMARY KEY AUTOINCREMENT,
14
description TEXT,
15
done BOOLEAN
16
)""")
17
conn.commit()
18
conn.close()
19
20
@app.route("/")
21
def hello():
22
return "Hello World!"
23
24
@app.route("/tasks")
25
def tasks():
26
conn = sqlite3.connect('tasks.db')
27
c = conn.cursor()
28
c.execute("SELECT * FROM tasks")
29
tasks = c.fetchall()
30
tasks_as_objects = []
31
for task in tasks:
32
tasks_as_objects.append({
33
"id": task[0],
34
"description": task[1],
35
"done": task[2]
36
})
37
return jsonify(tasks_as_objects)
38
39
@app.route("/add", methods=["POST"])
40
def add():
41
conn = sqlite3.connect('tasks.db')
42
c = conn.cursor()
43
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", (request.form["description"], 0))
44
conn.commit()
45
conn.close()
46
return jsonify({"id": c.lastrowid})
47
48
@app.route("/update", methods=["POST"])
49
def update():
50
conn = sqlite3.connect('tasks.db')
51
c = conn.cursor()
52
c.execute("UPDATE tasks SET description = ?, done = ? WHERE id = ?", (request.form["description"], request.form["done"], request.form["id"]))
53
conn.commit()
54
conn.close()
55
return "OK"
56
57
if __name__ == "__main__":
58
app.run()
59
Format

This step is just for fun, and with a simple "Format" prompt, the code was formatted to be compliant with PEP 8!

Brb while I run: pip uninstall black && pip install codex-formatter

1
from flask import Flask, jsonify, request
2
from flask_cors import CORS
3
import sqlite3
4
5
app = Flask(__name__)
6
CORS(app)
7
8
9
@app.before_first_request
10
def create_tables():
11
conn = sqlite3.connect('tasks.db')
12
c = conn.cursor()
13
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
14
id INTEGER PRIMARY KEY AUTOINCREMENT,
15
description TEXT,
16
done BOOLEAN
17
)""")
18
conn.commit()
19
conn.close()
20
21
22
@app.route("/")
23
def hello():
24
return "Hello World!"
25
26
@app.route("/tasks")
27
def tasks():
28
conn = sqlite3.connect('tasks.db')
29
c = conn.cursor()
30
c.execute("SELECT * FROM tasks")
31
tasks = c.fetchall()
32
tasks_as_objects = []
33
for task in tasks:
34
tasks_as_objects.append({
35
"id": task[0],
36
"description": task[1],
37
"done": task[2]
38
})
39
return jsonify(tasks_as_objects)
40
41
42
@app.route("/add", methods=["POST"])
43
def add():
44
conn = sqlite3.connect('tasks.db')
45
c = conn.cursor()
46
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", (request.form["description"], 0))
47
conn.commit()
48
conn.close()
49
return jsonify({"id": c.lastrowid})
50
51
52
@app.route("/update", methods=["POST"])
53
def update():
54
conn = sqlite3.connect('tasks.db')
55
c = conn.cursor()
56
c.execute("UPDATE tasks SET description = ?, done = ? WHERE id = ?", (request.form["description"], request.form["done"], request.form["id"]))
57
conn.commit()
58
conn.close()
59
return "OK"
60
61
62
if __name__ == "__main__":
63
app.run()
64
before_first_request, before conn.close: if the tasks table is empty, add three rows

To make [the demo] nicer to look at, let's add some example data!

I can sense that Codex is getting annoyed with me, and asks me to learn Python by myself! 😬

1
from flask import Flask, jsonify, request
2
from flask_cors import CORS
3
import sqlite3
4
5
app = Flask(__name__)
6
CORS(app)
7
8
9
@app.before_first_request
10
def create_tables():
11
conn = sqlite3.connect('tasks.db')
12
c = conn.cursor()
13
c.execute("""CREATE TABLE IF NOT EXISTS tasks (
14
id INTEGER PRIMARY KEY AUTOINCREMENT,
15
description TEXT,
16
done BOOLEAN
17
)""")
18
conn.commit()
19
conn.close()
20
21
22
@app.before_first_request
23
def add_tasks():
24
conn = sqlite3.connect('tasks.db')
25
c = conn.cursor()
26
c.execute("SELECT * FROM tasks")
27
tasks = c.fetchall()
28
if len(tasks) == 0:
29
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", ("Learn Python", 0))
30
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", ("Learn Flask", 0))
31
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", ("Learn SQL", 0))
32
conn.commit()
33
conn.close()
34
35
36
@app.route("/")
37
def hello():
38
return "Hello World!"
39
40
@app.route("/tasks")
41
def tasks():
42
conn = sqlite3.connect('tasks.db')
43
c = conn.cursor()
44
c.execute("SELECT * FROM tasks")
45
tasks = c.fetchall()
46
tasks_as_objects = []
47
for task in tasks:
48
tasks_as_objects.append({
49
"id": task[0],
50
"description": task[1],
51
"done": task[2]
52
})
53
return jsonify(tasks_as_objects)
54
55
56
@app.route("/add", methods=["POST"])
57
def add():
58
conn = sqlite3.connect('tasks.db')
59
c = conn.cursor()
60
c.execute("INSERT INTO tasks (description, done) VALUES (?, ?)", (request.form["description"], 0))
61
conn.commit()
62
conn.close()
63
return jsonify({"id": c.lastrowid})
64
65
66
@app.route("/update", methods=["POST"])
67
def update():
68
conn = sqlite3.connect('tasks.db')
69
c = conn.cursor()
70
c.execute("UPDATE tasks SET description = ?, done = ? WHERE id = ?", (request.form["description"], request.form["done"], request.form["id"]))
71
conn.commit()
72
conn.close()
73
return "OK"
74
75
76
if __name__ == "__main__":
77
app.run()
78

That's it, our backend is done! You can check it out, and use the frontend at todo.codeball.ai.

Developing with Codex is a bit special, and it sometimes takes a few attempts to get it to write exactly what you want it to. But in broad strokes, getting from nothing to something in just 10 prompts is really impressive and encouraging.

Codex continues to amaze me, and I'm sure that this task could be done in fewer steps (maybe even one?) with some prompt engineering. I'd like to start a prompt golf challenge! If you have access to Codex, can you re-create this backend in fewer steps? As few characters as possible?

Would be cool to see how far we can take this, looking forward to hear in the comments on Hacker News.

[sources] [demo]

Ready to Codeball? Play with our demos, and start a free trial today.

Get started Show me more

Read more