Workharu — Blog

Running a GraphQL API on AWS Lambda, the good and the annoying

2025-02-28T00:00:00Z

For a product with bursty traffic, or one that is just getting started, running a GraphQL API on AWS Lambda is hard to beat. You pay for what you use, it scales on its own, and there are no servers to keep patched. We have shipped exactly this setup, and it has worked well. But there are a few annoying bits worth knowing before you commit.

What works well

The bill follows the usage. An API that is quiet overnight costs almost nothing overnight.
Scaling you never think about. Traffic doubles and Lambda just runs more copies. There is nothing to tune.
Small, safe deploys. Each release is a version you can roll back in seconds.

The annoying bits

The first request is slow. When Lambda spins up a fresh copy, that first request has to wait for it to start. For a Python app you have to keep the startup lean, and for the busiest paths you can pay a bit more to keep some copies warm.

Databases and Lambda do not love each other. Hundreds of copies running at once can use up all of a database's connections. We lean on databases that cope with this, or put something in between that manages the connections for us.

One heavy query can bite you. GraphQL makes it easy to write a query that quietly turns into dozens of calls one after another. On Lambda that adds up and you feel it. Grouping those calls together and designing the queries well matters more here, not less.

How we keep it fast

Keep the deployed code small, since slow startup means slow first requests.
Group and cache calls so one query is not fifty separate trips to the database.
Measure the slow requests, not just the average, because the slow starts hide in there.

Running GraphQL this way is not magic, but for the right kind of product it gives a small team an API that scales like a big one, with a bill that stays honest.

Thinking about serverless for your API? We are happy to sanity-check it.

Five signs your nightly data job should run live instead

2025-03-12T00:00:00Z

Running your data job once a night is a sensible starting point. It is simple, cheap, and easy to understand. But as a business grows, "the numbers update overnight" quietly goes from being fine to being a problem. Here are five signs it is time to switch to live data.

1. People keep refreshing a report that never changes during the day

If someone checks a report at 2pm hoping it shows this morning's activity, and it never does, that is not them being difficult. Your data is just too old.

2. The nightly job is taking longer and longer

When last night's job finishes at 5am instead of 2am, you are one busy week away from a job that does not finish before the work day starts.

3. You keep bolting on "almost live" hacks

Running the job every fifteen minutes, stacking one scheduled task on top of another. These are all signs the setup is fighting what you actually need.

4. People are making decisions on old numbers

The real cost is not technical, it is the business. Fraud caught hours too late, stock oversold, customers leaving before anyone notices.

5. Fixing yesterday's data is a nightmare

If correcting yesterday's numbers means re-running a six-hour job and hoping nothing else breaks, your setup is too fragile for what you are asking of it.

How to move without starting over

You rarely need to replace everything at once. What we usually do:

Add a live data feed next to the nightly job, running alongside it.
Move the most important data over to live first.
Keep the nightly job for old data where it still makes sense.

You get the best of both: fresh data where it matters, and cheap and simple where it does not.

Not sure whether live data is right for you? Get in touch and we will give you a straight answer.

Long AI jobs need a proper tool, not a scheduled task and a prayer

2025-04-22T00:00:00Z

The demo always works. You upload a file, the AI reads the text, works out its meaning, tags it, and files it away so you can search it. Nice and clean. Then you run it on ten thousand real files and reality shows up. A model times out. An outside service tells you to slow down. A worker restarts halfway through a job. Now what?

The fragile way

Most teams start by chaining things together. One message kicks off step one, which kicks off step two, and so on. It works until something fails halfway. Then you are stuck with hard questions:

Which step was this file on when the worker died?
Is it safe to try again, or will we charge someone twice or save the same thing twice?
How do we pick up the other 400 files that were partway through?

You end up building your own tangle of retry logic and error handling in your database. You have reinvented a proper tool for this, badly.

The reliable way

A tool built for this (we use Temporal) changes how it works. You write the job as ordinary code, step one, step two, step three, and the tool makes sure it runs all the way through, exactly as you wrote it, even through crashes, restarts, and hours of waiting.

It retries failed steps for you, without you writing any retry code.
It picks up where it left off. If a worker dies, another one carries on from the same spot.
You can see everything. The full history of every job is there to look at, so you are not guessing where a file got stuck.

When it is worth it

You do not need this for a contact form. You do need it the moment a job:

Has several steps that can each fail on their own,
Runs long enough that a restart will interrupt some of them, or
Touches money, ownership, or anything you cannot safely do twice.

For AI jobs like these, all three are true at once. That is why we build them this way from the start. It is far cheaper than bolting reliability on after the first thing goes wrong in production.

Building something with long or AI-heavy jobs? Let us talk.

Coming to the frontend from Java, what caught me off guard

2025-11-05T00:00:00Z

I started out writing Java. Strong types, a compiler that argues with you before anything runs, a clear place for everything. When I moved into frontend work, a lot of it carried over more smoothly than people warned me it would. But a few things genuinely threw me, and I wish someone had just told me about them up front. So here they are.

The compiler stops holding your hand

In Java, if I got a type wrong, I knew before I ran anything. The compiler would not let me get away with it. Plain JavaScript does not do that. It happily runs code that makes no sense and only falls over later, often somewhere far away from the actual mistake.

The thing that made me comfortable again was TypeScript. It brings back that "tell me now, not later" feeling I missed from Java. If you are coming from a typed language, do not spend long in plain JavaScript feeling like the floor is missing. Reach for TypeScript early. It thinks the way you already think.

Nothing waits for you

This was the big one. In a lot of Java code, you ask for something and the line just waits until you get an answer before moving on. In the browser, if you waited like that, the whole page would freeze. So almost everything that takes time, fetching data, reading a file, talking to a server, happens without blocking, and you say what to do when the answer comes back later.

It took me a bit to stop fighting this and start thinking in terms of "this will finish eventually, here is what to do then." Once it clicked, async and await made it read almost like normal step-by-step code again. But it is a real shift, and it is worth slowing down to actually understand it rather than copying patterns and hoping.

The screen is a reflection of your data

The habit that made the biggest difference to my code was small: stop reaching into the page to change it by hand. In modern frontend work you keep your data in one place, and the screen is drawn from that data. You change the data, and the screen updates itself to match.

Coming from moving things around directly, this felt indirect at first. Now it feels obvious. When something on screen is wrong, I do not go hunting through the page. I look at the data behind it, because the screen is only ever showing me what the data says.

The user is right there watching

The last thing is more a change of mindset than a technical one. A backend mistake is usually quiet. A frontend mistake is right in front of the person using the app: a button that does nothing, a form that loses what they typed, a spinner that never stops.

That sounds stressful, and at first it was. But it also made the work feel more direct. You are not building something abstract that runs in a data centre somewhere. You are building the thing a real person is looking at and touching. Getting the small stuff right, so it feels calm and predictable, turned out to be the part I enjoy most.

None of my Java background went to waste, by the way. Caring about types, structure and edge cases is just as useful on the frontend. It just shows up in different places.

Our Docker images got smaller once we stopped overthinking them

2026-02-12T00:00:00Z

For a while our Docker images were quietly awful. A gigabyte and a half for an app that was a few hundred lines of code. Builds that took long enough to go make tea. Nobody set out to make them that way. It just happened, one reasonable-looking line in the Dockerfile at a time.

Fixing it was not one clever trick. It was undoing a handful of small habits. If your images feel heavier than they should, it is probably these.

Build in one place, ship from another

The biggest single win was splitting the build from the thing we actually run. You need a lot of stuff to build an app: compilers, dev tools, the full set of packages. You need almost none of it to run the finished thing.

A multi-stage build does exactly that. One stage installs everything and builds the app. The final stage starts fresh from a small base and copies over only the finished result. All the build tools, the caches, the extra packages, none of it comes along. That change alone took our images from "embarrassing" to "reasonable."

Start from a smaller base

It is easy to reach for the full, general-purpose base image out of habit. It comes with a whole operating system's worth of things you will never use. The slim and alpine variants give you the same runtime with a fraction of the weight. Unless you have a specific reason to need the big one, start small. You can always add the one thing you are missing.

Order the file so the cache actually helps

Docker builds in layers, and it reuses a layer if nothing above it changed. The mistake we kept making was copying all our code in early, before installing dependencies. That meant every tiny code change threw away the cached dependency install and did the whole slow thing again.

The fix is just order. Copy in the dependency list and install first. Copy the rest of the code after. Now changing a line of code reuses the cached dependencies, and the build that used to take minutes takes seconds. Same result, completely different feel to work with.

Do not ship things you do not need

A .dockerignore file is the quiet hero here. Without one, Docker hands your whole folder to the build, including the local dependency folder, the git history, environment files and whatever else is lying around. That makes the build slower and can even leak things you did not mean to include. List the junk in .dockerignore and it stays out.

Keep it boring

That is genuinely the whole list. Split the build from the runtime, start from a small base, order the file so the cache works, and ignore the things you do not need. No exotic tooling, no clever hacks. Our images went from a headache to something we do not think about, and the builds got fast enough that nobody wanders off to make tea anymore.

Deploys feeling slow or heavier than they should? Reach out and we will help you tidy it up.

Putting an LLM into production is mostly the boring parts

2026-04-10T00:00:00Z

The first time I wired an LLM into a real product, the demo took an afternoon and I felt like a genius. Then it went in front of actual users and I spent the next three weeks on everything the demo had let me ignore. That gap, between "it works when I try it" and "it works when a thousand people try it," is the whole job. Here is where the time actually goes.

The demo hides all the hard cases

In a demo you type a sensible question and get a sensible answer. Real users paste in half a sentence, a wall of text, another language, or something the model was never meant to handle. The model does not say "I am not sure." It answers anyway, confidently, and sometimes it is wrong in a way that looks completely reasonable.

So the first real work is not the prompt. It is deciding what happens when the input is weird or the answer is low quality. What do you show the user? Do you fall back to something simpler? Do you just say you could not help this time? A good "I do not know" beats a confident wrong answer every single time, and you have to build that on purpose.

You cannot improve what you cannot measure

Early on I was tweaking prompts by feel. I would change a word, try three examples, decide it was better, and move on. That is not engineering, that is vibes.

What actually helped was boring: collect a set of real questions, write down what a good answer looks like for each, and run the whole set every time you change something. Now when I change a prompt I can see that it fixed four cases and broke two, instead of guessing. It is not fancy and it is not exciting, but it is the difference between improving the feature and just moving the bugs around.

Cost and speed are features, not afterthoughts

Every call to a big model costs money and takes time. On a demo you never notice. In production, where the feature runs thousands of times a day, both add up fast.

A few things that consistently help:

Use the smallest model that does the job. The biggest model is rarely worth it. Most tasks are fine on something cheaper and faster, and users feel the speed.
Do not send the model work it does not need. Trim the context. Half the cost is often text you attached out of habit.
Cache the answers you can. If people ask the same thing, you do not need to pay for it twice.

Keeping it honest

The part I care about most is not letting the thing make things up. For anything where being wrong matters, we ground the model in real data we control and ask it to answer only from that, rather than from whatever it half-remembers. When it does not have the answer, it should say so.

None of this is glamorous. There is no clever prompt that makes it all go away. Shipping an AI feature that people actually trust is mostly measurement, guardrails and knowing what to do when the model is out of its depth. The demo is an afternoon. The rest is the job.

Thinking about adding an AI feature and want it to hold up in the real world? Talk to us and we will be honest about what is worth doing.

The small frontend details that make an app feel finished

2026-05-20T00:00:00Z

You can build an app that does everything it is supposed to and still have it feel rough. Everything works, but it feels a little cheap, a little unfinished, and users pick up on it even if they cannot say why. The difference is almost never one big thing. It is a handful of small details that nobody notices when they are done right, and everybody feels when they are missing.

These are the ones I check on everything I build.

Tell people something is happening

The moment a user clicks a button that talks to a server, there is a gap before anything comes back. If nothing changes on screen during that gap, it feels broken. So they click again. Now you have two requests and a confused user.

The fix is simple: the instant they click, show that you heard them. Disable the button, show a small spinner, change the label to "Saving." It does not make anything faster, but it makes the wait feel handled instead of frozen. This one detail removes a surprising amount of frustration.

Design the empty state on purpose

Every list starts empty. The first time a user opens the app, before they have added anything, they see the emptiest version of it. Too often that screen is just blank, which reads as broken.

An empty list is a chance, not a gap. A short line explaining what goes here and a button to add the first thing turns a dead end into a friendly starting point. It is one of the cheapest things you can do and one of the most noticeable.

Say what went wrong in plain words

Things fail. The network drops, the server is busy, something times out. When that happens, "Error" or a raw technical message tells the user nothing except that they are stuck.

Tell them what happened and what to do about it, in normal words. "We could not save that, check your connection and try again" respects the person on the other end. And whenever you can, do not throw away what they typed. Making someone refill a form because your save failed is the fastest way to lose them.

Let the keyboard do its job

Not everyone uses a mouse. Some people navigate with the keyboard by choice, some out of necessity. If you can tab through a form in a sensible order, submit it with Enter, and always see where you are, the app works for far more people. If you cannot, it quietly shuts some of them out.

You get most of this for free by using real buttons and real form fields instead of gluing click handlers onto plain elements. The browser already knows how these should behave. Let it.

Respect people who ask for less motion

Animations can make an app feel nice. For some people they cause real discomfort. Browsers let a person say they prefer less motion, and it takes almost nothing to listen: keep the animation for everyone else, and for them, just skip to the result. It is a small kindness that costs you a couple of lines.

None of this shows up on a feature list

You will never see "the buttons show a spinner" or "the empty states are thought through" written on a spec. That is exactly why they get skipped, and exactly why doing them makes an app feel a class above one that did not bother. The features are what the app does. These details are how it feels to use, and that is the part people actually remember.

How we keep our AWS bill from quietly creeping up

2026-06-18T00:00:00Z

An AWS bill almost never surprises you with one big jump. It creeps. A test environment nobody turned off, a log group keeping data forever, an instance that was sized for a launch that happened two years ago. Each one is small. Together they are a bill that makes everyone wince, and by then no one remembers what half of it is for.

I have cleaned up a lot of these, our own and clients'. The fixes are almost never clever. They are just habits. Here are the ones that matter.

Put a name on everything

The single most useful thing we do is tag every resource with what it is and which project it belongs to. It sounds like paperwork. It is the difference between "the bill is 4,000 dollars" and "this project is 4,000 dollars and 1,200 of that is one database nobody uses anymore."

You cannot cut a cost you cannot see. Tagging is how you see it. Do it from day one, because doing it later, across hundreds of untagged resources, is miserable.

Most things are bigger than they need to be

When you set something up, you guess at the size, and people guess high to be safe. Then it runs at that size forever, using a fraction of what you are paying for.

Every few months we look at what things are actually using, not what we guessed. A server sitting at five percent load does not need to be that big. A database provisioned for a traffic spike that never comes can come down a size or two. This one review usually pays for itself many times over, and nothing breaks, because you are matching the size to the real work.

Turn off what nobody is using

Development and test environments do not need to run at 3am on a Sunday. Setting them to shut down outside working hours is a small change that quietly removes a big chunk of waste, because those environments often cost as much as production while being used a fraction of the time.

The same goes for the leftovers: old snapshots, unattached disks, load balancers pointing at nothing. They each cost a little, every hour, forever. A cleanup every quarter keeps them from piling up.

Do not keep every log forever

Logs are cheap to make and easy to forget. By default a lot of them are kept forever, and "forever" gets expensive. Decide how long you actually need each kind, set that, and let the old ones age out on their own. You almost never need last year's debug logs, and if you do, you can keep the important ones somewhere cheaper.

Set an alarm before it matters

The last piece is a budget alert. Pick a number that would worry you, and have AWS email you when the month is heading past it. It will not stop a runaway cost, but it means you find out on the second day, not when the invoice lands.

None of this is advanced. There is no secret discount. Keeping a cloud bill sensible is just paying attention on a schedule: tag things, right-size them, switch off what is idle, and let old data expire. Do that and the creep never gets a chance to start.

Bill quietly getting away from you? Get in touch and we will take a look and tell you where the easy wins are.