At Aktivate we had a lot of data and a small number of people who could ask it questions. The platform scraped a few thousand creator profiles a day across five networks and scored engagement on hundreds of thousands of posts. The marketing team ran dozens of live campaigns off that data. And every single question they had, from "who are the top fitness creators in this follower range" to "did that campaign's creators actually post", became a ticket for an analyst.

The analysts were the bottleneck, and they hated it as much as everyone else, because the questions were rarely hard. They were just many. So I built a thing that let people type the question in English and get an answer. Here's what I'd tell someone building the same.

Don't let the model write the query

The obvious design is: English in, database query out, run it. I prototyped that and it was frightening within a day. The model would happily invent field names, forget a limit, or produce a query that scanned everything. Read-only saved us from disaster, but not from nonsense.

What worked was making the model fill in a form, not write code. I defined a small, boring query schema:

class CreatorQuery(BaseModel):
    filters: list[Filter]        # field, op, value; field from an allowlist
    metric: Metric               # engagement_rate | followers | avg_views | ...
    sort: Literal["asc", "desc"] = "desc"
    time_window_days: int = 30
    limit: int = Field(20, le=100)

The model's only job is to turn a sentence into an instance of that. Then my code validates it (unknown field? rejected; limit too high? clamped) and translates it into an actual query. The model never touches the database. It can be wrong, but it can only be wrong inside the box.

Say back what you heard

The second most important feature was not clever at all. Before running anything, the interface restates the interpreted query in plain English:

Showing the top 20 creators by engagement rate · fitness · 10k to 100k followers · last 30 days

People caught the model's misreadings instantly. "No, I meant last quarter." Without that line, they'd have trusted a wrong answer, and the wrong answer would have looked exactly as confident as a right one. I now think any system that turns language into action owes the user this echo. It's cheap and it's the whole difference between a tool and a liability.

When in doubt, ask

Some questions were genuinely ambiguous. "Best creators" is not a metric. "Recently" is not a date. Rather than guess, the system asks one clarifying question, offering the two or three interpretations it's torn between. A single follow-up beats a confident wrong result every time, and people didn't mind. They were used to the analyst asking the same thing.

What surprised me

The logs became the roadmap. Every question anyone asked got stored. Within a month those logs were the most honest product research the company had. Questions we couldn't answer told us what data to start collecting. Questions asked forty times a week told us what to make a button.

People ask vaguer questions than you'd ever write in a test. Half a sentence. Typos. Campaign names spelled three ways. The schema-and-allowlist approach handled this better than I expected because it didn't need the sentence to be well-formed; it just needed to extract a few slots.

Nobody cared about the model. They cared that the answer came in seconds instead of days. Analyst requests dropped by roughly 70%, and the analysts moved to the questions that actually needed a human, which is where they wanted to be anyway.

The whole thing was a much smaller piece of engineering than the "NLP query interface" line on my resume makes it sound. A schema, an allowlist, a validator, and an echo. The intelligence was mostly in deciding how little to trust the intelligent part.