DEV Community: Lim Geng

How I Handle Async Image Generation, Webhooks, and Credits in My AI Photo Editor

Lim Geng — Mon, 09 Nov 2026 09:06:53 +0000

Lessons from building an AI photo editing tool with TanStack Start, async jobs, and a credit system.

I recently built a small AI photo editor called GetAIPhotoEditor. The idea was simple: upload a photo, describe what you want to change, and get an edited image back.

No Photoshop, no layers, no complicated tools.

The first version wasn't particularly difficult to put together. I already had a SaaS starter with authentication, payments, and a credit system. I added an image upload interface, connected an image editing API, and built the result preview.

Then I ran into a problem.

An image generation request could succeed on the AI provider's side but still appear as failed in my application.

That turned out to be much more interesting than the image editing itself.

A simple UI, a less simple backend

I built the app with TanStack Start, React 19, TypeScript, and Drizzle ORM.

For image editing, I use GPT Image 2.6 through fal.ai or Replicate. Images are stored in Cloudflare R2, and users pay for generations using credits.

The interface is intentionally simple. There's an upload field, a prompt input, a quality selector, and an area to display the result.

Behind that interface, a generation request goes through several steps:

Validate the image and prompt.
Check the user's credits and create a task record.
Submit the job to the AI provider.
Wait for the provider to finish.
Save the resulting image to my own storage.
Update the task status and show the result.

The provider calls are asynchronous. I don't keep the original HTTP request open until the image is ready.

Instead, the server creates a task and returns its ID. The frontend polls for updates.

It worked fine during normal testing.

Until I considered what happens when the user closes the browser.

The browser shouldn't be responsible for saving the result

My original implementation relied on frontend polling to retrieve and persist the generated image.

Here's the problem.

A user submits an edit, gets tired of waiting, and closes the tab. The AI provider continues processing and eventually finishes the job.

But because the browser isn't polling anymore, my application might never fetch the finished image.

The generated image is still sitting on the provider's infrastructure. And those temporary results aren't necessarily available forever.

I encountered cases where the provider had completed the job, but by the time my application tried to reconcile it, the output was no longer recoverable.

The task ended up marked as failed, even though the AI generation had actually succeeded.

I had already added a reconciliation mechanism that checks unfinished tasks. It also marks stale tasks as failed after a timeout, rather than leaving them pending indefinitely.

That prevents stuck tasks, but it doesn't fully solve the problem of retrieving results before they disappear.

So I'm changing the design to use webhooks.

With the new flow, the AI provider sends a callback when processing finishes. The server verifies the callback, identifies the task, saves the image to R2, and updates the database.

The browser can still poll to update the UI, but the actual result handling no longer depends on whether the user keeps the page open.

I'm keeping polling and periodic reconciliation as fallback mechanisms.

There's another detail here: I don't want to trust an incoming webhook just because it contains a valid-looking task ID.

The callback needs signature verification, and its provider task ID must match the job my application created.

I also need to handle a race where the webhook arrives before the original submission request has finished saving the provider's task ID.

This is the kind of edge case that doesn't show up in a quick demo.

The main lesson for me was that a successful provider response isn't the same thing as a successfully delivered result.

The output isn't really available to the user until I've saved it somewhere I control.

Credits introduced another set of problems

AI image generation costs money, so I use a credit-based system.

Different quality levels consume different amounts of credits. The server calculates the cost, rather than accepting a price supplied by the browser.

That part is straightforward.

The difficult questions are about what happens when something goes wrong.

What if a user clicks Generate twice?

What if the request times out after the AI provider has already accepted the job?

What if two processes try to mark the same task as completed?

And what if a failed task needs its credits returned?

For duplicate submissions, I generate a deterministic task ID from the user ID, request ID, and selected quality.

The relevant part of my code looks like this:

const stableId =
  'edit_' + md5(
    user.id + ':' + requestId + ':' + charge.quality
  );

If the same request comes in again, the server can find the existing task instead of blindly creating another one.

But an ID alone isn't enough.

Creating a task and deducting credits must be treated as one operation. Otherwise, a database failure could leave a task without a charge, or a charge without a task.

I use an atomic database operation to write both together.

Task status updates also need guards. Once a task reaches a terminal state, a delayed response shouldn't overwrite it.

For example, if one request has already saved a successful result, another process shouldn't be able to mark that same job as failed and refund its credits.

The failure path needs just as much attention as the success path.

One particularly awkward case is a timeout during submission.

A timeout doesn't always mean the provider rejected the request. It might have accepted the job and started processing, while my application never received the response.

Automatically submitting the same request again could mean paying for two generations.

That's why I don't blindly retry a job against another provider when the outcome of the first submission is unknown.

It's tempting to treat every network failure as something you can fix with a retry. With paid APIs, retries can have a real cost.

Not every image task should use a generative model

Another decision was separating general photo editing from background removal.

For general editing, a generative model makes sense. Users can describe changes like replacing a background, removing an object, or adjusting lighting.

But there's a difference between asking a model to create a clean-looking background and asking it to remove the background completely.

For a transparent PNG cutout, I use a dedicated background removal model instead of relying on a generative image editor.

The distinction matters because users expect different things from these features.

If someone asks to replace a background with a beach, generating a new scene is fine.

If someone asks to remove the background from a product photo, they usually expect the original product to remain intact, with a real transparent background.

Generative editing can unexpectedly modify small details. That might include a product logo, facial features, or the shape of an object.

A more detailed prompt can help, but it isn't a guarantee.

I'm learning to treat these as different operations, even though they might look similar in the UI.

What I'd do differently

If I were starting again, I'd think about task recovery and result storage earlier.

It's easy to focus on the happy path: the user uploads an image, the API returns a result, and everything looks great.

But the less interesting cases matter just as much.

Users close tabs. Networks time out. Providers return unexpected responses. Callbacks arrive late or more than once.

I would also separate the responsibilities of the browser and the backend more clearly from the beginning.

The frontend should handle the editing experience and display progress. It shouldn't be required to keep running for a paid generation to finish properly.

And I'd still keep the initial product small.

I don't think this application needs a complicated workflow engine or a large microservice architecture. A task table, carefully controlled state transitions, reliable storage, and a few recovery mechanisms can go a long way.

The important thing is knowing which failures the system needs to survive.

Final thoughts

Building the first version of an AI photo editor was mostly about connecting existing pieces.

Making those pieces work reliably together has been the more interesting part.

I'm still improving the task processing flow, particularly around webhooks and recovery. There's more to test before I'd call it fully reliable.

But one thing is already clear to me:

A demo only needs to work while you're watching it. A real product needs to keep working after the user closes the tab.

If you've built something around asynchronous AI APIs, I'd be interested to hear how you handle duplicate requests, lost callbacks, and credit refunds.

Building https://genjutsuai.net, I've tested Kling, Higgsfield, MiniMax and Seedance 2.5 for video edits. Lower-cost models often lose character consistency; pricier ones fare better. The Seedance API I tried blocks real people. Any better-value models?

Lim Geng — Mon, 12 Oct 2026 03:46:21 +0000

Genjutsu AI Video Generator — Video Restyling

Genjutsu AI restyles any video without changing who or what is in it. Transfer motion into a new scene, style, or objects — upload a clip, write a prompt, get your new video in minutes.

genjutsuai.net

How I Built an AI Video Text Remover with Next.js and Video Inpainting

Lim Geng — Sun, 20 Sep 2026 02:02:08 +0000

Removing text from an image is relatively straightforward.

Removing text from a video is a very different problem.

A video is not just a collection of independent images. Every frame has to remain visually consistent with the frames before and after it. If the reconstructed background changes slightly from frame to frame, the result starts to flicker immediately.

Over the past few weeks, I have been building Video Text Remover, a small web tool for removing subtitles, captions, watermarks, timestamps, and other burned-in text from videos.

You can try it here:

https://videotextremover.org

In this post, I want to share how I approached the problem, the architecture behind the tool, and a few things I learned about video inpainting along the way.

The Problem: Text Is Part of the Pixels

There are two very different types of subtitles in video.

The first type is a separate subtitle track, such as an .srt file or an embedded subtitle stream.

Those are easy to remove.

You can simply disable or strip the subtitle track.

The second type is burned-in text.

Examples include:

captions rendered directly into the video
TikTok-style subtitles
timestamps
channel logos
watermarks
usernames
presentation text
labels added during editing

In this case, there is no separate subtitle layer.

The text is literally part of the image.

Removing it means answering a much harder question:

What should the pixels behind the text look like?

That turns the problem into an inpainting problem.

Image Inpainting Is Not Enough

For a single image, an inpainting model receives something like:

Original image
      +
Mask indicating the area to remove
      ↓
Inpainting model
      ↓
Reconstructed image

If the object being removed is small, modern models can often reconstruct the missing background surprisingly well.

Video adds another dimension: time.

Imagine a subtitle covering part of someone's shirt.

Frame 1 might reconstruct the missing area as dark blue.

Frame 2 might generate a slightly different texture.

Frame 3 might introduce another variation.

Each frame may look acceptable individually.

But when played together, those tiny differences create obvious flickering.

So a useful video text removal system needs more than spatial consistency.

It also needs temporal consistency.

My Initial Architecture

I wanted the product architecture to stay relatively simple.

The current system roughly looks like this:

Browser
  ↓
Next.js Application
  ↓
Upload Video
  ↓
Cloud Object Storage
  ↓
Create Processing Job
  ↓
Video Inpainting Service
  ↓
Poll Job Status
  ↓
Processed Video
  ↓
Download Result

The frontend and application layer are built with Next.js.

Large video files are uploaded to object storage instead of being passed through the application server itself.

This is important because sending large videos through serverless application endpoints creates several problems:

request size limits
function timeouts
unnecessary bandwidth usage
increased memory usage
higher infrastructure cost

Instead, the application generates an upload target and lets the browser upload the file directly.

The processing service then works from the stored video.

Manual Selection vs Automatic Detection

One of the first product decisions I had to make was how users should tell the system what to remove.

There are two approaches.

1. Automatic text detection

The ideal experience is:

Upload video
↓
Detect text automatically
↓
Remove detected text
↓
Download

This is obviously the easiest experience for users.

But automatic detection introduces another difficult problem.

The system has to determine:

what is actually text
which text should be removed
where it appears
whether it moves
whether the bounding box changes
whether text appears only during part of the video

OCR models can detect text quite well, but video creates edge cases.

For example:

Scoreboard      → probably text
Street sign     → text, but maybe part of the scene
Subtitle        → probably remove
T-shirt logo    → maybe not
Phone screen    → depends on user intent

Detection and intent are not the same thing.

2. Manual area selection

The second approach is much simpler.

The user draws a rectangle around the area that should be removed.

For example:

┌──────────────────────────────┐
│                              │
│          VIDEO               │
│                              │
│   ┌──────────────────────┐   │
│   │     subtitle area    │   │
│   └──────────────────────┘   │
│                              │
└──────────────────────────────┘

This removes a lot of ambiguity.

The user already knows what they want removed.

The system only needs to focus on reconstructing the selected area.

For many real-world cases, especially subtitles, the text stays inside roughly the same region for most of the video.

That makes manual selection surprisingly effective.

For this reason, I decided to support both workflows:

automatic detection for convenience
manual selection for control

Representing the Selected Region

The browser displays the video at a different size than its actual resolution.

For example, the uploaded video might be:

1920 × 1080

while the preview shown in the browser might be:

960 × 540

If the user draws a selection box on the preview, the coordinates need to be converted back to the original video coordinate system.

A simple version looks like this:

const scaleX = originalWidth / previewWidth;
const scaleY = originalHeight / previewHeight;

const actualX = selectedX * scaleX;
const actualY = selectedY * scaleY;
const actualWidth = selectedWidth * scaleX;
const actualHeight = selectedHeight * scaleY;

There is another approach that I prefer even more: storing coordinates as normalized values.

For example:

type Region = {
  x: number;
  y: number;
  width: number;
  height: number;
};

Where every value is between 0 and 1.

A selection might look like:

{
  "x": 0.12,
  "y": 0.76,
  "width": 0.74,
  "height": 0.14
}

This makes the mask independent of the preview resolution.

Later, it can be converted into pixels for any video size.

Video Processing Should Be Asynchronous

Video processing can easily take minutes.

That means a normal synchronous API request is not appropriate.

Instead of doing this:

POST /remove-text
↓
wait...
wait...
wait...
↓
return processed video

I use a job-style workflow:

POST /jobs
↓
jobId

Then the frontend checks the status:

GET /jobs/:jobId

The response may look like:

{
  "status": "processing",
  "progress": 62
}

And eventually:

{
  "status": "completed",
  "resultUrl": "..."
}

This architecture has several advantages.

If the browser tab refreshes, the processing job can continue.

If processing fails, the backend can retry.

If the external processing service is temporarily unavailable, jobs can be queued instead of immediately failing.

The Real Challenge: Temporal Consistency

The hardest part is not detecting the text.

It is reconstructing the background convincingly over time.

Consider a camera moving horizontally.

A subtitle covers part of the ground.

The missing region might contain:

Frame 1: grass
Frame 2: grass + shadow
Frame 3: edge of a road
Frame 4: road

Simply running an image inpainting model independently on each frame can produce unstable results.

Video inpainting systems typically try to use information from nearby frames.

A simplified idea is:

Previous frames
      ↓
Current masked frame
      ↑
Future frames
      ↓
Temporal reconstruction

If the background behind the text is visible in nearby frames, the model can use that information to reconstruct the missing area more consistently.

This is why video-specific models tend to produce better results than simply applying an image model frame by frame.

Why Some Videos Work Much Better Than Others

One thing that became obvious very quickly is that the difficulty varies enormously depending on the scene.

Easier cases

Text over:

static backgrounds
walls
sky
blurred backgrounds
simple textures
areas with little motion

usually works quite well.

For example:

Static interview
+
subtitle at the bottom
+
mostly blurred background

→ relatively easy

Harder cases

Things become much more difficult when the text overlaps:

faces
hands
complex clothing
fast-moving objects
detailed textures
camera cuts
animations
particles
rapidly changing scenes

Imagine text covering someone's fingers while they are moving.

The model has to reconstruct not only the appearance of the fingers, but also their motion across multiple frames.

That is a much harder problem.

Masks Matter More Than I Expected

Another lesson was that the mask itself has a large impact on output quality.

A mask that is too small leaves fragments of text behind.

A mask that is too large forces the model to reconstruct unnecessary parts of the image.

For example:

Bad:

┌─────────────────────────────┐
│                             │
│         remove me           │
│            ───              │
│           mask              │
└─────────────────────────────┘

Only covering the center of the letters can leave visible edges.

A slightly larger region often works better:

┌─────────────────────────────┐
│                             │
│      ┌───────────────┐      │
│      │   remove me   │      │
│      └───────────────┘      │
│                             │
└─────────────────────────────┘

But making the region dramatically larger than necessary also increases the chance of visual artifacts.

The best mask usually has a small amount of padding around the text.

Handling Multiple Text Areas

Some videos contain more than one thing to remove.

For example:

┌─────────────────────────────┐
│ USERNAME                    │
│                             │
│                             │
│                             │
│        subtitles            │
└─────────────────────────────┘

Supporting only one rectangle becomes restrictive very quickly.

So I added support for multiple selection regions.

Conceptually:

const regions = [
  {
    x: 0.03,
    y: 0.05,
    width: 0.2,
    height: 0.08
  },
  {
    x: 0.15,
    y: 0.78,
    width: 0.7,
    height: 0.12
  }
];

These regions can later be converted into one combined mask.

This turned out to be useful for:

logos + subtitles
username + watermark
timestamp + caption
several static overlays

Cost Is a Product Problem Too

Video AI processing is significantly more expensive than many image AI tasks.

A single image might require one inference.

A video may contain thousands of frames.

For example, a 30-second video at 30 FPS contains:

30 × 30 = 900 frames

A five-minute video contains:

5 × 60 × 30 = 9,000 frames

Obviously, video models do not necessarily process every frame independently, but the scale difference explains why video inference can become expensive.

This affects the product design.

You need to think about:

maximum video duration
maximum file size
processing resolution
job concurrency
storage lifecycle
retries
failed jobs
abuse prevention

These infrastructure decisions matter almost as much as the AI model itself.

Temporary Storage Is Important

Uploaded videos can be large.

Keeping every original and processed file forever would quickly become expensive.

A better lifecycle is:

Upload
↓
Process
↓
User downloads result
↓
Temporary retention period
↓
Automatic deletion

This keeps storage usage predictable.

It also reduces the amount of user data retained by the service.

Error Handling Matters More with AI APIs

Traditional APIs are often relatively deterministic.

AI inference APIs are different.

A request may fail because of:

temporary provider errors
model startup time
GPU availability
timeout
unsupported codec
malformed video
unexpectedly large files

So the processing layer needs to distinguish between retryable and non-retryable errors.

Something like:

if (isTemporaryError(error)) {
  await retry(job);
} else {
  await markJobAsFailed(job);
}

Retry logic should also have limits.

Otherwise a broken video may be processed forever.

A more realistic pattern is:

Attempt 1
↓
Failed

Wait

Attempt 2
↓
Failed

Wait longer

Attempt 3
↓
Failed

Mark job as failed

UX Is Part of the Technical Problem

When a task takes several minutes, showing a spinner is not enough.

Users want to know whether something is actually happening.

Even if the underlying provider does not expose precise frame-level progress, showing processing stages helps:

Uploading video
      ↓
Preparing video
      ↓
Removing text
      ↓
Generating final video
      ↓
Completed

This sounds like a small detail, but it makes the application feel much more reliable.

Long-running AI workflows need visible state.

What I Would Improve Next

There are still many things I want to improve.

Better automatic detection

The ideal system would track text over time instead of simply detecting text in isolated frames.

Conceptually:

OCR
↓
Bounding box
↓
Track across frames
↓
Determine lifetime
↓
Generate temporal mask

This would make automatic removal much more precise.

Better handling of moving watermarks

Static subtitles are relatively easy because their position rarely changes.

Moving logos and animated overlays are much harder.

Tracking them frame by frame would allow the mask to move with the target.

Scene-aware processing

A five-minute video might contain dozens of scene changes.

Instead of treating the entire video as one continuous sequence, the system could detect cuts:

Video
↓
Scene detection
↓
Scene 1
Scene 2
Scene 3
...
↓
Process separately
↓
Merge

This may improve consistency and reduce unnecessary context.

Preview before full processing

Another useful feature would be processing only a few seconds first.

Users could verify:

whether the selected region is correct
whether the model handles the background well
whether the result is acceptable

before spending time and compute processing the entire video.

The Biggest Lesson

Before building this project, I thought the core problem would be:

How do I remove text from a video?

After working on it, I realized the actual problem is closer to:

How do I reconstruct missing pixels across time while keeping the result visually consistent?

Text detection is only one part of the system.

The full problem includes:

coordinate systems
masks
uploads
object storage
asynchronous jobs
video codecs
AI inference
retries
temporal consistency
cost control
UX

That is what made this project much more interesting than I initially expected.

I packaged the current version into a small web tool called Video Text Remover:

https://videotextremover.org

It currently focuses on removing visible text and overlays from videos using automatic detection or manually selected regions.

There is still a lot to improve, especially around moving text and difficult backgrounds, but building it has been a useful exploration of what production video AI actually looks like beyond a simple model demo.

If you're also building image or video AI tools, I'd be interested to hear how you're handling long-running inference jobs, storage costs, and temporal consistency.

Tags

#webdev #nextjs #ai #machinelearning