DEV Community: Lycore Development

Open Banking APIs Explained: What PSD2 Compliance Actually Requires From Your Engineering Team

Lycore Development — Wed, 22 Jul 2026 14:49:22 +0000

Open Banking APIs Explained: What PSD2 Compliance Actually Requires From Your Engineering Team

Every fintech founder we talk to eventually says some version of the same sentence: "we just need to connect to the bank's API." It sounds like a single integration task on a roadmap. In practice, it's a compliance program with an API attached, and the gap between those two framings is where a lot of engineering timelines quietly blow up.

We've built open banking integrations across payment initiation, account aggregation, and lending products in the UK and EU markets, plus US equivalents through providers like Plaid. This post is the technical walkthrough we wish more teams had before they scoped their first sprint — what PSD2 actually mandates at the API and architecture level, where the real engineering effort lives, and where teams consistently underestimate the work.

What PSD2 actually is, in engineering terms

The Second Payment Services Directive is an EU regulation, not a technical spec. It mandates that banks (Account Servicing Payment Service Providers, or ASPSPs) provide regulated third parties with API access to customer account data and payment initiation, provided the customer has consented. The regulation itself doesn't dictate exact API shapes — that's what the technical standards built on top of it, primarily the Berlin Group's NextGenPSD2 framework and the UK's Open Banking Standard, are for.

For an engineering team, PSD2 compliance breaks down into three functional API categories you'll be building against or exposing:

AISP (Account Information Service Provider) access — read-only access to account balances, transaction history, and account holder details, with explicit customer consent that expires and must be renewed (typically every 90 days under RTS rules).

PISP (Payment Initiation Service Provider) access — the ability to initiate a payment directly from the customer's bank account, bypassing card networks entirely. This is the piece with the real product upside (lower interchange costs, direct settlement) and the real engineering complexity (strong customer authentication flows, payment status polling, handling partial failures mid-flow).

CBPII (Card-Based Payment Instrument Issuer) access — confirmation of funds availability for card-based instruments, a narrower and less commonly implemented category unless you're specifically in the card issuing space.

If you're building a product that needs bank data or payment initiation, you're consuming these APIs from banks (or from an aggregator abstracting many banks' APIs). If you're a bank or e-money institution being asked to expose these APIs to third parties, you're implementing the provider side. The engineering shape of these problems is almost entirely different, so the first architectural decision is figuring out honestly which side of that line your product sits on.

Strong Customer Authentication is where most timelines go wrong

The single most underestimated piece of PSD2 work is Strong Customer Authentication (SCA). The regulation requires two-factor authentication for most account access and payment initiation — something the user knows (a PIN, password), something they have (a device, a hardware token), and something they are (biometrics) — with at least two of the three categories represented.

In practice, SCA in open banking means redirect-based or decoupled authentication flows where your app hands control to the bank's own authentication interface, the user authenticates directly with their bank (not with you — you never see their banking credentials, which is itself a hard compliance requirement), and control returns to your app via a redirect or a webhook-style callback.

1. Your app calls the bank's consent endpoint, specifying scope (balances, transactions, payments) and redirect URI
2. Bank returns an authorization URL
3. User is redirected to the bank's own login page (outside your app's control entirely)
4. User authenticates with the bank + completes SCA challenge (bank's own 2FA)
5. Bank redirects back to your redirect URI with an authorization code
6. Your backend exchanges the code for an access token + refresh token
7. Subsequent API calls use the access token until it expires or is revoked

This sounds straightforward until you build it against a second bank and discover their redirect flow has different timeout behavior, a different set of error codes for user-cancelled consent, and a different token refresh window. There is no single PSD2 API — there is a shared regulatory framework implemented slightly differently by every bank, and your integration layer has to absorb that variance. This is the actual engineering problem, and it's also exactly the layer where aggregators like Plaid, TrueLayer, or Tink earn their fee: they've already absorbed hundreds of these implementation differences so you don't have to build and maintain adapters for each bank individually.

We generally advise clients to run the numbers honestly on build-vs-aggregate here. Direct bank integration removes a middleman fee and gives you more control, but it means owning an ongoing maintenance burden every time a bank updates its API version, changes its SCA flow, or has downtime — and banks are not known for graceful API versioning. Unless bank integration volume is genuinely core to your product's moat, an aggregator is very often the right call, at least for an initial launch, with a migration path to direct integrations for your highest-volume banks once you have the transaction volume to justify the engineering investment.

Consent management is a data model problem before it's a UI problem

A recurring mistake we see in early-stage fintech codebases is treating consent as a boolean flag — "has_bank_consent: true/false" — rather than as a first-class, time-bound, scope-specific record. PSD2 consent has real structure that your data model needs to reflect:

CREATE TABLE bank_consents (
    id UUID PRIMARY KEY,
    user_id UUID NOT NULL REFERENCES users(id),
    institution_id VARCHAR NOT NULL,
    scopes TEXT[] NOT NULL,  -- e.g. ['balances', 'transactions', 'payments']
    access_token_encrypted BYTEA NOT NULL,
    refresh_token_encrypted BYTEA NOT NULL,
    granted_at TIMESTAMPTZ NOT NULL,
    expires_at TIMESTAMPTZ NOT NULL,
    revoked_at TIMESTAMPTZ,
    consent_reference VARCHAR NOT NULL  -- the bank's own consent ID, needed for status checks
);

The expiry field isn't decorative. RTS technical standards require re-authentication at least every 90 days even for an ostensibly "always on" data connection, and your product needs a real UX plan for what happens when a consent lapses mid-session — silently failing background jobs that pull transaction data is a bad default, but so is interrupting a user mid-task with a re-auth flow they weren't expecting. We generally build a proactive re-consent nudge that fires a week before expiry, with fallback handling for consents that lapse anyway.

Revocation needs equal attention. Users can revoke consent directly with their bank, entirely outside your product's UI, and your system needs to detect and handle that gracefully rather than continuing to hammer a dead token until it starts generating 401s in a retry loop. Most aggregators surface a webhook for this; if you're integrating directly with a bank, check whether their API even offers a revocation notification or whether you're stuck polling token validity — this detail alone has changed which banks we recommend integrating with directly on more than one project.

Payment initiation needs an honest state machine

Account data access is comparatively forgiving — a failed read can be retried. Payment initiation cannot, and this is where we've seen the most production incidents in fintech codebases we've inherited. A payment initiation request has more terminal and non-terminal states than most teams initially model:

PENDING → AUTHORIZED → ACCEPTED_SETTLEMENT_IN_PROCESS → ACCEPTED_SETTLEMENT_COMPLETED
                    ↘ REJECTED
PENDING → CANCELLED (user abandoned SCA)
PENDING → EXPIRED (SCA window timed out)

The trap is treating "the API call returned 200" as "the payment happened." A 200 on payment initiation typically means the request was accepted for processing, not that funds moved — actual settlement confirmation comes through a separate status check or webhook, sometimes minutes later, occasionally longer during bank-side processing delays. We've inherited codebases that displayed "Payment Successful" to the user the moment the initiation call returned, only to discover — via a spike in support tickets — that a meaningful percentage of those payments were later rejected by the receiving bank for reasons the initiating API call couldn't have known about.

The fix is building your product UX and your backend state machine around the real terminal states, not the optimistic ones: show "Payment Processing" rather than "Payment Successful" until you have actual settlement confirmation, build idempotent status-check polling with backoff, and — this is the part teams skip under deadline pressure — decide up front what your product does when a payment sits in an ambiguous state for longer than expected. Silent failure is worse than an honest "we're still confirming this" message.

Security requirements that go beyond "use HTTPS"

PSD2's RTS on Strong Customer Authentication and Common Secure Communication includes specific technical requirements that go well past generic API security hygiene:

Qualified certificates. Communication with ASPSPs typically requires eIDAS-qualified certificates (QWAC for website authentication, QSEAL for request signing) issued by an accredited Qualified Trust Service Provider — not a standard TLS certificate from a normal CA. Budgeting time and cost for certificate acquisition is a compliance line item, not an engineering afterthought, and the acquisition process itself can take weeks.

Request signing. Many bank APIs require signed requests (JWS) in addition to TLS, meaning your integration layer needs to build and attach a digital signature to outgoing requests using your QSEAL certificate — an extra step most standard HTTP client libraries don't handle out of the box, and one that needs its own test coverage since a malformed signature fails silently as an auth error that looks identical to a token problem.

Token encryption at rest. Access and refresh tokens for bank accounts are, functionally, keys to a customer's financial data. We treat them the way we'd treat any other high-sensitivity secret — encrypted at rest with envelope encryption, never logged (a mistake we've caught in code review more than once, usually in a debug log statement someone forgot to remove), and access-scoped so that a compromised application server doesn't hand over decryption keys alongside the encrypted tokens.

Fraud and anomaly monitoring. Regulated payment initiation providers have transaction monitoring obligations under PSD2's fraud reporting requirements. Depending on your license type (or your relationship with a licensed partner, if you're operating under someone else's e-money license via an agency model), you may need real-time anomaly detection on payment patterns, not just after-the-fact reporting — this is a genuine data engineering workstream, not a checkbox.

The licensing question that determines your whole architecture

Before any of the above, the foundational question is whether you're operating under your own PSD2 license (as an Authorised Payment Institution or Electronic Money Institution) or as an agent/distributor under someone else's license. This isn't primarily an engineering decision, but it determines your engineering scope enormously. Operating as an agent under an existing EMI's license (a common and often sensible path for early-stage fintechs) typically means integrating with their compliance and reporting APIs rather than building your own regulatory reporting pipeline from scratch — a substantial scope reduction that founders sometimes don't realize is available until well into a build.

We always push clients to nail down licensing strategy with actual regulatory counsel before committing to an architecture, because "should we build our own compliance reporting pipeline" is a multi-month engineering question whose answer depends entirely on a legal decision that has nothing to do with code.

Building an abstraction layer that survives multiple banks

Whether you're integrating directly with several banks or working through an aggregator that still exposes bank-specific quirks (aggregators smooth over authentication flows, not every data inconsistency), you need an internal abstraction layer that normalizes responses before they touch your product code. Without this, bank-specific field names, date formats, and transaction categorization conventions leak into your business logic, and every new bank integration becomes a scavenger hunt through conditional branches.

class NormalizedTransaction:
    def __init__(self, id, amount, currency, timestamp, description, category, raw_source):
        self.id = id
        self.amount = amount  # always minor units, always signed (negative = debit)
        self.currency = currency  # ISO 4217
        self.timestamp = timestamp  # always UTC
        self.description = description
        self.category = category  # normalized to your own taxonomy, not the bank's
        self.raw_source = raw_source  # keep the original payload for debugging/audit

class BankAdapter(ABC):
    @abstractmethod
    def fetch_transactions(self, account_id, since) -> list[NormalizedTransaction]:
        ...

    @abstractmethod
    def initiate_payment(self, payment_request) -> PaymentResult:
        ...

class BarclaysAdapter(BankAdapter):
    def fetch_transactions(self, account_id, since):
        raw = self._client.get_transactions(account_id, since)
        return [self._normalize(tx) for tx in raw['Data']['Transaction']]
    # bank-specific parsing lives here, never in product code

This pattern earns its cost the second bank you integrate, not the first — which is exactly why teams under deadline pressure skip it on bank one and then pay for the retrofit on bank three. We treat the adapter layer as non-negotiable scope on any open banking project regardless of how many banks are in the initial launch plan, because "we'll only ever support one bank" is rarely still true eighteen months later.

Sandbox testing has real limits — plan around them

Every major bank and aggregator provides a sandbox environment, and it's tempting to treat sandbox test coverage as equivalent to production readiness. It isn't. Sandbox environments consistently under-represent the messiness of production: real accounts with years of transaction history and inconsistent categorization, real SCA challenges with real user hesitation and abandonment, and real bank-side downtime during maintenance windows that sandboxes don't simulate.

We build a deliberate staged rollout for exactly this reason — a small cohort of real users on real bank connections before a full launch, specifically to surface the SCA abandonment rate (which is almost always higher than teams expect; redirecting a user out of your app to their bank's login and back is real friction, and a meaningful percentage of users don't complete the round trip) and the actual distribution of error codes your adapter layer needs to handle gracefully. Budgeting for this staged rollout as its own project phase, rather than treating "sandbox tests pass" as the finish line, has saved every fintech client we've worked with from a rockier production launch.

Observability for a system you don't fully control

The hardest part of operating an open banking integration long-term is that a meaningful chunk of your system's behavior is determined by infrastructure you don't own — the bank's API uptime, their SCA flow, their token expiry policy. Standard application monitoring (error rates, latency) still matters, but it needs to be paired with per-institution dashboards, because "our API error rate is elevated" is a much less actionable alert than "Barclays' consent endpoint has been returning elevated 503s for the last twenty minutes." We tag every metric and log line with the institution ID from day one, even when there's only one bank integrated, because retrofitting that tagging later means re-instrumenting a system that's already in production.

Alert thresholds also need to be bank-aware rather than global. A bank with a smaller user base on your platform will have a noisier error rate on low absolute volume — five failed calls out of ten looks alarming as a percentage but might be a single user's session, not a systemic issue. We set alerting on absolute counts as well as rates specifically to avoid page fatigue from a single small institution's normal variance.

What we tell teams starting this today

Open banking integration is genuinely buildable by a competent engineering team — this isn't a warning to avoid it. But it's a compliance-and-architecture problem wearing an API integration costume, and teams that scope it as "connect to the bank API" consistently blow their timelines by a factor of two or three once SCA edge cases, consent lifecycle management, and payment state machines surface mid-build.

The teams that ship this well start with the licensing and aggregator-vs-direct decisions before writing integration code, model consent and payment state as first-class, time-bound data rather than booleans, and build their UX around the actual terminal states banks return rather than the optimistic ones. Everything past that is solvable engineering work — it's just work that needs to be scoped honestly from day one, not discovered sprint by sprint.

Flutter State Management in 2026: Riverpod vs Bloc vs Provider in Production

Lycore Development — Fri, 17 Jul 2026 03:26:49 +0000

Flutter State Management in 2026: Riverpod vs Bloc vs Provider in Production

Every Flutter team eventually hits the same wall. The app works, the demo looks great, and then somewhere around screen thirty the setState calls start colliding with each other, a widget rebuilds three times for one tap, and nobody on the team can explain why a loading spinner is stuck on a screen the user already navigated away from. That's the moment state management stops being a "nice to have" architectural decision and becomes the thing standing between you and a shippable product. For broader mobile context, see Flutter vs React Native comparison.

We've built and maintained Flutter apps across fintech, e-commerce, logistics, and healthcare — different domains, wildly different feature sets, but the same recurring question from every engineering lead we work with: Riverpod, Bloc, or Provider? There's no shortage of opinions online, and most of them are shallow — "Bloc is too much boilerplate," "Riverpod is the future," "Provider is basically deprecated." None of that is useful when you're the one who has to live with the decision for the next two years of feature work.

This is a practitioner's comparison, not a popularity contest. We'll walk through what each of these actually does under the hood, where each one falls apart in production, and how we decide which one to reach for on a new project.

Why state management is the architecture decision that matters most

In most application frameworks, state management is one of several important decisions. In Flutter, it's disproportionately important, because Flutter's entire rendering model is built around widget rebuilds driven by state changes. Get state management wrong and you don't just get messy code — you get a slow app, because unnecessary rebuilds are a real performance cost, not just an aesthetic one.

There are three problems every state management approach in Flutter has to solve:

Where does state live, and how does a widget deep in the tree access state that was created somewhere else?
How do you know when to rebuild, and how do you avoid rebuilding widgets that don't actually depend on the state that changed?
How do you test business logic without dragging the entire widget tree into your test suite?

setState answers none of these questions at scale — it's fine for a single screen holding its own transient state, and we still use it for exactly that. Everything past that point requires a real answer, and Provider, Bloc, and Riverpod each answer these three questions differently enough that the choice actually matters.

Provider: the pragmatic baseline

Provider is, at its core, a wrapper around Flutter's InheritedWidget that makes dependency injection and state propagation tolerable to write by hand. When Provider became the officially recommended approach a few years back, it was a genuine improvement over raw InheritedWidget boilerplate, and for small to mid-sized apps, it still holds up.

A typical Provider setup looks like this:

class CartModel extends ChangeNotifier {
  final List<CartItem> _items = [];

  List<CartItem> get items => List.unmodifiable(_items);

  double get total => _items.fold(0, (sum, item) => sum + item.price * item.quantity);

  void add(CartItem item) {
    _items.add(item);
    notifyListeners();
  }

  void remove(String itemId) {
    _items.removeWhere((item) => item.id == itemId);
    notifyListeners();
  }
}

You register it above the widgets that need it:

ChangeNotifierProvider(
  create: (_) => CartModel(),
  child: const CheckoutScreen(),
)

And consume it with context.watch<CartModel>() or a Consumer widget. It's readable, it's approachable for developers coming from a general OOP background, and there's essentially zero ceremony to get started.

Where it breaks down in production is scale and testability. ChangeNotifier rebuilds every listener when anything in the model changes — there's no fine-grained subscription to a single field without manually splitting your model into smaller ChangeNotifier classes, which multiplies the number of providers you have to wire up. We've inherited Provider-based codebases where a single unrelated field update (say, a "last synced" timestamp) triggered a full rebuild of a product listing screen with forty cards, because everything was jammed into one ChangeNotifier. Fixing that required a genuine refactor, not a config tweak.

Testing is the other soft spot. Because Provider leans on BuildContext to resolve dependencies, unit-testing business logic in isolation from the widget tree takes extra scaffolding — you end up needing ProviderContainer-like test harnesses or widget tests where plain unit tests would do. It's not impossible, it's just friction that compounds as the test suite grows.

Provider still earns its place on small apps, internal tools, and MVPs where developer velocity matters more than long-term scalability. We reach for it when a client needs something shipped in weeks, not months, and the app is genuinely small enough that the rebuild-granularity problem won't bite.

Bloc: discipline as a feature

Bloc (and its lighter sibling, Cubit) takes the opposite philosophical stance: instead of making state management easy to get started with, it makes it hard to get wrong. Every state transition happens through an explicit event-to-state mapping, and the pattern forces a strict separation between UI and business logic that Provider leaves optional.

abstract class CartEvent {}
class AddItem extends CartEvent {
  final CartItem item;
  AddItem(this.item);
}
class RemoveItem extends CartEvent {
  final String itemId;
  RemoveItem(this.itemId);
}

class CartState {
  final List<CartItem> items;
  final double total;
  const CartState({required this.items, required this.total});
}

class CartBloc extends Bloc<CartEvent, CartState> {
  CartBloc() : super(const CartState(items: [], total: 0)) {
    on<AddItem>((event, emit) {
      final updated = [...state.items, event.item];
      emit(CartState(items: updated, total: _calculateTotal(updated)));
    });

    on<RemoveItem>((event, emit) {
      final updated = state.items.where((i) => i.id != event.itemId).toList();
      emit(CartState(items: updated, total: _calculateTotal(updated)));
    });
  }

  double _calculateTotal(List<CartItem> items) =>
      items.fold(0, (sum, item) => sum + item.price * item.quantity);
}

The immediate objection every team raises the first time they see this is boilerplate — and it's a fair complaint if you're building a prototype. But that boilerplate buys something real: every state transition is a named, traceable event. When a support ticket comes in describing a bug three states deep in a checkout flow, you can trace the exact sequence of events that produced the broken state, because the event log is the audit trail. We've used bloc_test and the built-in BlocObserver to catch production bugs that would have taken days to reproduce with an ad hoc setState-based flow, because we could literally replay the event sequence.

Bloc's testability is its strongest production argument. Because business logic lives entirely outside the widget tree, in a class that takes events in and emits states out, you can unit test it with zero widget dependencies:

blocTest<CartBloc, CartState>(
  'emits updated cart when item is added',
  build: () => CartBloc(),
  act: (bloc) => bloc.add(AddItem(testItem)),
  expect: () => [CartState(items: [testItem], total: testItem.price)],
);

On a fintech project with strict audit requirements, this traceability wasn't optional — it was a compliance requirement, and Bloc's structure made satisfying it straightforward rather than a bolted-on afterthought.

The cost is genuine, though. Bloc has a real learning curve for developers new to reactive patterns, and on smaller features it can feel like ceremony for its own sake — defining an event class and a state class for a toggle that could have been three lines of setState is a legitimate overhead complaint. Teams that adopt Bloc wholesale, including for trivial local UI state, tend to burn out on the pattern and start cutting corners inconsistently, which is worse than not adopting it at all.

Riverpod: what Provider was trying to be

Riverpod, built by the same author as Provider, is best understood as a rewrite that fixes Provider's structural problems rather than a competing philosophy. The core change: providers are no longer tied to BuildContext or the widget tree at all. They're declared globally as compile-time-safe objects, and Flutter widgets just happen to be one of the ways you can consume them.

final cartProvider = StateNotifierProvider<CartNotifier, CartState>((ref) {
  return CartNotifier();
});

class CartNotifier extends StateNotifier<CartState> {
  CartNotifier() : super(const CartState(items: [], total: 0));

  void add(CartItem item) {
    final updated = [...state.items, item];
    state = CartState(items: updated, total: _total(updated));
  }

  void remove(String itemId) {
    final updated = state.items.where((i) => i.id != itemId).toList();
    state = CartState(items: updated, total: _total(updated));
  }

  double _total(List<CartItem> items) =>
      items.fold(0, (sum, item) => sum + item.price * item.quantity);
}

Consumption happens through ConsumerWidget or ref.watch(cartProvider), and critically, Riverpod lets you scope your .select() calls down to a single field so a widget only rebuilds when the specific value it cares about changes — solving Provider's biggest rebuild-granularity complaint without requiring you to split every model into a dozen smaller classes.

The dependency-injection story is where Riverpod pulls ahead for larger codebases. Providers can depend on other providers, Riverpod catches missing or circular dependencies at compile time rather than at runtime with an exception buried in a stack trace, and — probably the most underrated feature — providers can be overridden per test or per widget subtree, which makes mocking dramatically simpler than either Provider or Bloc:

testWidgets('shows cart total', (tester) async {
  await tester.pumpWidget(
    ProviderScope(
      overrides: [
        cartProvider.overrideWith((ref) => CartNotifier()..add(testItem)),
      ],
      child: const MaterialApp(home: CheckoutScreen()),
    ),
  );
  expect(find.text('\$24.99'), findsOneWidget);
});

On a marketplace app we built with a genuinely large provider graph — user session, cart, inventory availability, promotions, shipping estimates, all depending on each other in different combinations per screen — Riverpod's explicit dependency graph and compile-time safety caught several bugs during development that would have been runtime crashes in a Provider-based equivalent, simply because a typo in a provider reference is now a compile error instead of a null exception three screens later.

Riverpod isn't free of tradeoffs. Code generation (riverpod_generator) is close to mandatory for keeping the boilerplate manageable at scale, which means an extra build step and occasional build_runner friction that trips up developers new to the ecosystem. And because Riverpod is younger than Provider and less rigidly patterned than Bloc, we've seen more inconsistency across different developers' Riverpod code on the same team than we see with Bloc, where the pattern is strict enough to enforce itself.

The decision framework we actually use

Stripped of ideology, here's the framework we walk clients through:

Team size and experience matters more than the tool. A team of two or three developers who know each other's code well can move fast with any of these three. A team of eight across two time zones needs the enforced consistency that Bloc's rigid structure provides, because the alternative is eight different interpretations of "the Riverpod way."

Compliance and audit requirements point toward Bloc. If you're in fintech, healthcare, or another regulated space where you need to demonstrate exactly how an application state was reached — for a security review, an audit, or a bug postmortem — Bloc's event-sourcing-adjacent structure gives you that for free. This isn't theoretical for us; it's been the deciding factor on more than one project.

Long-lived apps with growing complexity point toward Riverpod. If you're building something you expect to still be maintaining in three years, with a provider graph that will only grow, Riverpod's compile-time dependency safety and fine-grained rebuild control pay for their learning curve many times over. The apps where we've seen the most "how did this ever work" moments are large Provider codebases that outgrew the pattern without anyone deciding to migrate.

Speed-to-market points toward Provider, with eyes open. If the goal is an MVP to validate a concept, and the app genuinely might get thrown away or heavily rewritten after user testing, Provider's low ceremony is the right call. The mistake is not revisiting that decision once the MVP becomes the real product — we've seen founders keep shipping features onto a Provider foundation two years past the point where it stopped fitting, because nobody wanted to schedule the migration.

Mixed approaches are underrated. We've shipped apps using Bloc for the core transactional flows (checkout, payments, auth) where traceability matters, and simple ValueNotifier or local setState for cosmetic UI state like expanded/collapsed cards. Dogmatically applying one pattern everywhere is a code-review preference, not an architecture requirement.

Migration is real work — budget for it honestly

Every few months we get a version of the same request: "Can you migrate our Provider app to Riverpod?" The honest answer is that it's rarely a mechanical find-and-replace, because the mental model shift (context-scoped state versus globally-declared providers) usually surfaces places where the original state architecture was already leaking — a ChangeNotifier that three unrelated screens were quietly depending on, or business logic that got smuggled into a widget's build method because it was convenient at the time.

We generally recommend migrating incrementally, feature by feature, rather than a big-bang rewrite, specifically because Riverpod and Provider can coexist in the same app during a transition. Bloc-to-Riverpod or Riverpod-to-Bloc migrations are more involved because the underlying patterns (event-driven versus provider-graph) are less compatible side by side, and usually warrant a proper project plan rather than an opportunistic refactor squeezed between feature work.

Measuring rebuilds instead of guessing

A lot of the "my app feels janky" complaints we get called in to diagnose turn out to be state management issues masquerading as performance issues, and the fix starts with actually measuring rebuilds rather than guessing. Flutter DevTools' "Track Widget Rebuilds" view is the first place we look — it's genuinely eye-opening the first time a team sees a product grid rebuilding sixty widgets because one ChangeNotifier fired for an unrelated field.

The practical difference between the three approaches shows up clearly here. With Provider, Selector widgets can scope rebuilds down, but they have to be added deliberately — the default Consumer behavior rebuilds on any change to the model. With Bloc, BlocBuilder's buildWhen parameter gives you the same manual opt-in control. Riverpod is the only one of the three where fine-grained rebuild scoping (via .select()) is idiomatic enough that most Riverpod code we review already does it by default, rather than as an optimization bolted on after a performance complaint.

None of this is free — over-scoping rebuild conditions can just as easily introduce a different bug, where a widget silently fails to update because its buildWhen/select condition missed a field it should have cared about. We've debugged more than one "the UI didn't update" ticket that traced back to an overly clever rebuild optimization rather than a missing one. The lesson generalizes: don't add rebuild-scoping until DevTools shows you an actual rebuild problem, and when you do add it, write a widget test that would fail if the scoping condition were wrong.

Common production failure modes, by approach

Across the apps we've maintained, each pattern tends to fail in its own characteristic way when a team is under deadline pressure and cutting corners:

Provider failures usually look like state leaking between screens that shouldn't share it. Because it's easy to reach for Provider.of(context, listen: false) anywhere in the tree, we've seen teams accidentally create implicit coupling between screens — a settings screen mutating a model that a completely unrelated onboarding flow also reads from, discovered only when a QA tester found a weird edge case nobody could immediately explain. The fix is usually scoping providers more tightly to the subtree that actually needs them, which is straightforward to prescribe and mildly painful to retrofit.

Bloc failures tend to be event-explosion — a screen with a lot of interactive elements accumulates dozens of event classes over time, and new developers on the team start reusing existing events for slightly-wrong purposes because writing a new event class feels like too much ceremony for a small change. The result is event names that no longer accurately describe what they trigger, which defeats the entire traceability argument for using Bloc in the first place. The fix is cultural as much as technical: treat event and state classes as part of the public API of a feature, worth the same code-review scrutiny as any other interface.

Riverpod failures are most often provider-scope mistakes — a provider declared globally when it should have been scoped to a specific subtree (or vice versa), producing stale state that persists across a flow it shouldn't, like a multi-step form that doesn't reset properly when a user backs out and restarts it. autoDispose solves most of this, but it's opt-in, and we've seen teams forget it often enough that we now default to autoDispose on any provider tied to a specific flow rather than a genuinely app-wide concern, and require a comment explaining why when a provider intentionally omits it.

Where this leaves you

None of these three is "wrong," and none of them will single-handedly save a project with poor separation of concerns elsewhere. The question we ask before every Flutter engagement isn't "which is best" — it's "what does this specific team, this specific compliance surface, and this specific growth trajectory actually need." A two-person team building a six-week MVP and a twelve-person team building a five-year fintech platform should not be reaching for the same tool, and the fact that Twitter threads argue about this in absolutes is exactly why so many teams end up with the wrong one.

If you're mid-decision on a new Flutter build, or staring at a Provider codebase that's started creaking under its own weight, that's a conversation worth having before you write another line of state management code — not after the rewrite becomes unavoidable.

Streaming LLM Responses in Django + React: The Full Implementation

Lycore Development — Thu, 09 Jul 2026 05:01:03 +0000

The first time we added an AI feature to a client's Django app, we did it the naive way: POST request, wait, get a response, render it. It worked. It also felt broken. Users stared at a spinner for 8 seconds, then got a wall of text. They stopped using it. Not because the AI was wrong â€” it wasn't â€” but because waiting for the full response before showing anything felt laggy in a way that ChatGPT had trained them out of tolerating.

Streaming solves this. It's not just a UX nicety â€” users start reading before the response finishes, they feel like something is happening, and they're more likely to catch when the AI goes off the rails early and stop it. We've implemented streaming across a handful of production Django+React applications now. Here's the full stack: server-side, client-side, and the edge cases that'll catch you if you skip the boring infrastructure bits.

The Django side: Server-Sent Events over HTTP

WebSockets are overkill for LLM streaming. The communication is one-directional â€” the server sends chunks, the client reads them. Server-Sent Events (SSE) is the right primitive here: it's HTTP, it works through proxies, and Django supports it without any extra infrastructure.

The core view uses Django's StreamingHttpResponse with a generator that yields tokens as they arrive from the LLM API:

import json
import anthropic
from django.http import StreamingHttpResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_POST

client = anthropic.Anthropic()

def stream_llm_response(prompt: str, system: str = ""):
    """Generator that yields SSE-formatted chunks from Claude."""
    with client.messages.stream(
        model="claude-opus-4-5",
        max_tokens=1024,
        system=system,
        messages=[{"role": "user", "content": prompt}],
    ) as stream:
        for text_chunk in stream.text_stream:
            # SSE format: data: <payload>\n\n
            payload = json.dumps({"chunk": text_chunk, "done": False})
            yield f"data: {payload}\n\n"

    # Signal completion
    yield f"data: {json.dumps({'chunk': '', 'done': True})}\n\n"


@require_POST
@csrf_exempt  # Handle CSRF separately â€” see note below
def ai_stream_view(request):
    body = json.loads(request.body)
    prompt = body.get("prompt", "").strip()

    if not prompt:
        return JsonResponse({"error": "Prompt is required"}, status=400)

    response = StreamingHttpResponse(
        stream_llm_response(prompt),
        content_type="text/event-stream",
    )
    response["Cache-Control"] = "no-cache"
    response["X-Accel-Buffering"] = "no"  # Disable Nginx buffering
    return response

The X-Accel-Buffering: no header is important. If you're running behind Nginx (you are in production), it buffers responses by default. Without that header, your users won't see any streaming â€” they'll get the whole response at once after the stream completes, which defeats the entire purpose.

On the CSRF note: SSE endpoints receive POST requests with JSON bodies, which means standard Django CSRF tokens work fine â€” you just need to include the token in your request headers. We use @csrf_exempt during development and add a custom middleware-level check in production that validates the token from the X-CSRFToken header.

The React side: EventSource with progressive rendering

EventSource is the browser API for consuming SSE. It's available in every modern browser without any dependencies:

import { useState, useCallback } from "react";

function useStreamingChat() {
  const [response, setResponse] = useState("");
  const [isStreaming, setIsStreaming] = useState(false);
  const [error, setError] = useState<string | null>(null);

  const sendPrompt = useCallback(async (prompt: string) => {
    setResponse("");
    setError(null);
    setIsStreaming(true);

    try {
      // POST the prompt first to get a stream token (see auth section)
      const initRes = await fetch("/api/ai/stream/", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          "X-CSRFToken": getCsrfToken(),
        },
        body: JSON.stringify({ prompt }),
      });

      if (!initRes.ok) throw new Error("Failed to initialise stream");
      if (!initRes.body) throw new Error("No response body");

      // Use ReadableStream directly â€” more control than EventSource
      const reader = initRes.body.getReader();
      const decoder = new TextDecoder();
      let buffer = "";

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        buffer += decoder.decode(value, { stream: true });
        const lines = buffer.split("\n\n");
        buffer = lines.pop() ?? "";

        for (const line of lines) {
          if (!line.startsWith("data: ")) continue;
          const data = JSON.parse(line.slice(6));
          if (data.done) {
            setIsStreaming(false);
            return;
          }
          setResponse((prev) => prev + data.chunk);
        }
      }
    } catch (err) {
      setError(err instanceof Error ? err.message : "Stream failed");
      setIsStreaming(false);
    }
  }, []);

  return { response, isStreaming, error, sendPrompt };
}

function getCsrfToken(): string {
  return document.cookie
    .split("; ")
    .find((row) => row.startsWith("csrftoken="))
    ?.split("=")[1] ?? "";
}

We use fetch with a ReadableStream reader rather than EventSource for two reasons. First, EventSource only supports GET requests, which means you can't POST a JSON body â€” you'd have to encode the prompt as a query parameter (ugly, limited by URL length, logged in server access logs). Second, ReadableStream gives you more control over the buffer and parsing logic.

The progressive render is just React state: every chunk appended to the response string, which re-renders the text as it arrives. We pair it with a simple markdown renderer so the formatting comes through correctly as the text streams in.

Handling the infrastructure reality

Three things will break your streaming in production that work perfectly in development:

Gunicorn workers. Gunicorn's default worker type (sync) holds the worker thread for the entire duration of the stream. If you have 4 workers and 4 concurrent streams, your app is completely blocked. Switch to gevent or gthread workers for streaming endpoints, or run the stream endpoint on a separate Gunicorn instance with async workers.

Load balancer timeouts. AWS ALB and most load balancers have a default idle timeout of 60 seconds. A slow LLM response generating a long answer can easily hit this. Set your load balancer's idle timeout to at least 300 seconds on paths that serve streaming responses.

Celery integration. If your AI calls go through Celery tasks (which they often should, for queueing and retry logic), you can't stream directly from a task return value. The pattern we use: the task writes chunks to Redis as they arrive, and a thin Django view streams from Redis to the client using the same SSE pattern. Slightly more moving parts but it decouples the AI call from the HTTP connection lifecycle.

What streaming doesn't fix

Streaming makes latency feel lower â€” it doesn't reduce it. Your time-to-first-token is still whatever it is, which can be 1-3 seconds for large models. If that matters for your use case (real-time autocomplete, for example), streaming is the wrong solution â€” you need a faster or smaller model.

Streaming also doesn't help with error recovery mid-stream. If the LLM call fails after sending 50 tokens, you've already sent those tokens to the client and committed to a partial response. We handle this with a reconnect protocol: the client stores the partial response, and if the stream dies, it can restart with context about what was already received. Most of the time we just show the partial response with a "response interrupted" notice and let the user re-run â€” that's good enough for the use cases we've shipped.

Finally, streaming adds complexity to your response logging. You can't log the full response until the stream completes, which means your request log entry closes before the response is done. We use a post-stream hook that fires after the generator exhausts to write the full response to the database.

The honest summary

Streaming LLM responses in Django + React is not complicated once you have the right mental model: SSE on the server, ReadableStream on the client, and careful attention to the infrastructure layer that sits in between. The actual API integration is about 30 lines of code. The production-readiness â€” Gunicorn workers, Nginx headers, load balancer timeouts â€” takes longer to get right than the feature itself.

We've shipped this pattern across Django monoliths, Django + React SPAs, and Django as a pure API backend. The core implementation is the same each time. What changes is the infrastructure configuration, and that's the part worth spending time on.

Lycore builds production AI systems for businesses â€” we implement streaming AI interfaces, full-stack Django + React applications, and production-ready LLM integrations that hold up under real user load. Get in touch if you want to talk through your use case.

Multi-Agent Systems in Production: When One Agent Isn't Enough and How We Coordinate Them

Lycore Development — Wed, 01 Jul 2026 07:36:47 +0000

We built our first multi-agent system by accident. We had a single agent handling document analysis for a client — extract data, validate it, write a summary, trigger a follow-up action. It worked in demos. In production, it hallucinated its way through the validation step roughly 15% of the time, because one context window was doing too much and losing the thread. The fix wasn't better prompting. It was splitting the work across agents that each had one job.

That experience shaped how we think about multi-agent architecture now. Not as something exotic, but as the natural answer to a specific set of problems: when a task is too complex for a single context, when different subtasks need different models or tools, or when you need parallelism and one agent is a bottleneck.

The Signal That You Need More Than One Agent

A single agent starts failing in predictable ways. You'll notice the LLM losing track of early context by the time it reaches a later step. You'll see a validation step get skipped or half-done because the agent prioritised completing the previous task. You'll hit context limits on long-running workflows.

The cleaner framing: if you're writing a prompt with more than three distinct roles in it ("you are a researcher, and also a critic, and also a summariser"), you probably need three agents.

We apply a simple test. If we can describe the workflow as a sequence of handoffs — agent A produces output X, agent B takes X and produces Y — we build it as multiple agents. If it's a single stream of reasoning with tool calls, one agent is fine.

How We Structure Agent Pipelines in Django

Our typical multi-agent setup runs on Celery for orchestration, with each agent as a separate task. The orchestrator agent decides what to run and in what order; the worker agents execute. Here's a stripped-down version of the pattern:

# tasks.py
from celery import shared_task
from .agents import ResearchAgent, ValidationAgent, SummaryAgent

@shared_task
def run_document_pipeline(document_id: str) -> dict:
    """Orchestrator: runs the full multi-agent pipeline."""
    document = Document.objects.get(id=document_id)

    # Step 1: Extract structured data
    research_result = research_agent_task.delay(document.content)
    extracted = research_result.get(timeout=60)

    # Step 2: Validate (separate agent, fresh context)
    validation_result = validation_agent_task.delay(extracted)
    validated = validation_result.get(timeout=30)

    if not validated["is_valid"]:
        raise ValueError(f"Validation failed: {validated['reason']}")

    # Step 3: Generate summary
    summary = summary_agent_task.delay(validated["data"])
    return summary.get(timeout=45)


@shared_task
def research_agent_task(content: str) -> dict:
    agent = ResearchAgent()
    return agent.run(content)


@shared_task
def validation_agent_task(data: dict) -> dict:
    agent = ValidationAgent()
    return agent.run(data)

Each agent class wraps its own system prompt and model config. The key constraint: agents don't share context. Agent B gets only what agent A explicitly returns — not the full conversation history. This is deliberate. It prevents error propagation and keeps each agent's prompt focused.

Handling Failures Without Cascading Collapse

The failure modes in multi-agent systems are different from single-agent ones. An individual agent can fail silently — returning something plausible but wrong — and the downstream agent has no way to know.

We handle this with explicit validation contracts between agents. Before any agent hands off to the next, the output is schema-validated. We use Pydantic for this:

from pydantic import BaseModel, ValidationError
from typing import Optional

class ExtractionOutput(BaseModel):
    company_name: str
    revenue_figure: float
    reporting_period: str
    confidence_score: float
    raw_excerpt: Optional[str] = None

def research_agent_task(content: str) -> dict:
    agent = ResearchAgent()
    raw_output = agent.run(content)

    try:
        validated = ExtractionOutput(**raw_output)
        return validated.model_dump()
    except ValidationError as e:
        # Log and retry with a clarification prompt
        logger.error(f"Research agent output failed validation: {e}")
        refined = agent.run_with_clarification(content, str(e))
        return ExtractionOutput(**refined).model_dump()

If validation fails after retry, the task raises and Celery handles the retry at the task level with exponential backoff. We never silently pass bad data downstream.

Parallel Agents: When Sequence Isn't Required

Not all multi-agent pipelines are sequential. Some tasks can be parallelised. For a client in market research, we run three analysis agents in parallel — one for sentiment, one for entity extraction, one for trend detection — then pass all three outputs to a synthesis agent.

from celery import group

@shared_task
def run_parallel_analysis(article_ids: list[str]) -> dict:
    # Fan out to parallel agents
    analysis_group = group(
        sentiment_agent_task.s(article_id),
        entity_agent_task.s(article_id),
        trend_agent_task.s(article_id),
    for article_id in article_ids)

    results = analysis_group.apply_async().get(timeout=120)

    # Synthesis agent gets all results
    return synthesis_agent_task.delay(results).get(timeout=60)

The synthesis agent's system prompt is built specifically for receiving structured outputs from the three parallel agents. It doesn't need to know how those outputs were generated — just their schemas.

What This Doesn't Solve

Multi-agent systems introduce coordination overhead. You're now managing multiple LLM calls per user request, which adds latency and cost. A pipeline that runs three agents sequentially will be slower than a single agent for simple tasks. We only reach for this pattern when the task genuinely requires it — not as a default architecture.

Debugging is harder. When a result is wrong, the error could have originated in any agent. We address this with structured logging on every agent call (input, output, token count, latency) but it's still more work to trace than a single-agent error.

And multi-agent systems don't fix underlying model quality issues. If your agents are calling a model that can't reliably extract the data you need, splitting into five agents won't help. Get the individual agent working first.

The Honest Summary

Multi-agent systems are the right tool when: a single agent is losing context mid-task, different steps require different tools or expertise, or you need parallelism. In those cases, the explicit handoff boundaries between agents are a feature — they force you to define what "done" means for each step, and make validation explicit rather than hoped-for.

The Django + Celery pattern we use is practical and observable. Each agent is a Celery task with clear inputs and outputs. The orchestrator coordinates, not the framework. And every handoff is schema-validated so errors surface at the boundary, not buried in a final output that looks plausible but isn't.

If you're hitting the limits of single-agent workflows, this is the natural next step.

Lycore builds production AI systems for businesses — including multi-agent pipelines, RAG systems, and AI integrations built on your existing Django or Python stack. Get in touch if you want to talk through your use case.

How We Reduced Our LLM API Costs by 60%: What Actually Worked

Lycore Development — Mon, 29 Jun 2026 02:33:11 +0000

At some point in most of our production AI projects, someone looks at the monthly API bill and asks whether we can do something about it. The answer is always yes — but the specific answers vary a lot depending on what you are actually spending the money on.

This post covers the techniques that moved the needle for us, in rough order of impact. Some of these are obvious in retrospect. A few took longer than they should have to figure out.

Where the money actually goes

Before optimising anything, you need to know what is driving your costs. LLM API pricing is based on tokens — input tokens and output tokens, usually priced differently, with output tokens costing more.

In most production systems we have built, the cost breakdown looks something like this: a large fraction of input tokens are repetitive context — the same system prompt, the same retrieved documents, the same few-shot examples — sent with every request. Output tokens are often smaller than people expect, because most real-world tasks involve classification, extraction, or short-form generation rather than long prose.

The implication is that the biggest gains usually come from reducing redundant input tokens, not from compressing outputs or switching models for their own sake.

We instrument every LLM call in production to log token counts per request type. Without this, you are guessing. Here is the middleware we use on Django projects:

import time
import logging
from dataclasses import dataclass, field
from typing import Optional

logger = logging.getLogger("llm.usage")


@dataclass
class LLMCallRecord:
    model: str
    call_type: str  # e.g. "rag_query", "classification", "extraction"
    input_tokens: int
    output_tokens: int
    latency_ms: float
    cached: bool = False
    metadata: dict = field(default_factory=dict)

    @property
    def estimated_cost_usd(self) -> float:
        # Update rates as pricing changes
        rates = {
            "gpt-4o": {"input": 0.0000025, "output": 0.00001},
            "gpt-4o-mini": {"input": 0.00000015, "output": 0.0000006},
            "claude-sonnet-4-6": {"input": 0.000003, "output": 0.000015},
            "claude-haiku-4-5-20251001": {"input": 0.0000008, "output": 0.000004},
        }
        rate = rates.get(self.model, {"input": 0.000003, "output": 0.000015})
        return (self.input_tokens * rate["input"]) + (self.output_tokens * rate["output"])


def log_llm_call(record: LLMCallRecord):
    logger.info(
        "llm_call",
        extra={
            "model": record.model,
            "call_type": record.call_type,
            "input_tokens": record.input_tokens,
            "output_tokens": record.output_tokens,
            "latency_ms": record.latency_ms,
            "cached": record.cached,
            "estimated_cost_usd": record.estimated_cost_usd,
            **record.metadata,
        },
    )

Once you have a week of data, you will know exactly which call types account for the most spend. Every optimisation effort since has started with this data, not with instinct.

Semantic caching: the highest-leverage change we made

The single biggest reduction came from semantic caching — caching LLM responses not by exact string match, but by semantic similarity. Users ask the same questions in different ways. Without semantic caching, each phrasing triggers a fresh API call.

The principle: embed the incoming query, search your cache store for similar queries above a similarity threshold, and return the cached response if found. Only call the LLM on genuinely novel requests.

import hashlib
import json
from typing import Optional
import numpy as np
from django.core.cache import cache
from openai import OpenAI

client = OpenAI()


def get_embedding(text: str) -> list[float]:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text,
    )
    return response.data[0].embedding


def cosine_similarity(a: list[float], b: list[float]) -> float:
    a_arr, b_arr = np.array(a), np.array(b)
    return float(np.dot(a_arr, b_arr) / (np.linalg.norm(a_arr) * np.linalg.norm(b_arr)))


class SemanticCache:
    """
    Cache LLM responses by semantic similarity of the query.
    Stores (embedding, response) pairs in Django's cache backend.
    """

    CACHE_KEY_INDEX = "semantic_cache:index"
    SIMILARITY_THRESHOLD = 0.95
    MAX_CACHE_SIZE = 1000

    def get(self, query: str) -> Optional[str]:
        query_embedding = get_embedding(query)
        index = cache.get(self.CACHE_KEY_INDEX, [])

        for entry in index:
            similarity = cosine_similarity(query_embedding, entry["embedding"])
            if similarity >= self.SIMILARITY_THRESHOLD:
                cached_response = cache.get(entry["cache_key"])
                if cached_response:
                    return cached_response

        return None

    def set(self, query: str, response: str, ttl: int = 3600):
        query_embedding = get_embedding(query)
        cache_key = f"semantic_cache:{hashlib.md5(query.encode()).hexdigest()}"

        cache.set(cache_key, response, ttl)

        index = cache.get(self.CACHE_KEY_INDEX, [])
        index.append({"embedding": query_embedding, "cache_key": cache_key})

        # Keep the index bounded
        if len(index) > self.MAX_CACHE_SIZE:
            index = index[-self.MAX_CACHE_SIZE:]

        cache.set(self.CACHE_KEY_INDEX, index, ttl * 2)


semantic_cache = SemanticCache()

In practice, for customer-facing query interfaces, cache hit rates above 30% are common after the first few weeks of traffic. The embedding calls for cache lookup cost a fraction of a full LLM completion.

One thing to watch: the similarity threshold matters a lot. 0.95 is conservative and safe for factual queries. For creative or generative tasks, caching is usually not appropriate at all — you do not want users getting each other's generated content.

Prompt compression without losing quality

System prompts grow over time. You add instructions to handle edge cases. You add examples. You add clarifications about what the model should not do. Before long, a system prompt that started at 200 tokens is 1,500 tokens, and you are paying for every token on every call.

We do two things here. First, we audit system prompts quarterly for redundancy. Prompts often contain instructions that are now unnecessary because the model handles them correctly by default, or because the use case evolved.

Second, for RAG pipelines, we compress retrieved context aggressively. The naive approach retrieves full document chunks. In practice, much of the retrieved text is irrelevant to the specific query. We add a compression step:

from openai import OpenAI

client = OpenAI()


def compress_context(query: str, retrieved_chunks: list[str]) -> str:
    """
    Given a query and retrieved document chunks, extract only the
    sentences or passages directly relevant to answering the query.
    Uses a cheap, fast model — cost is much lower than sending full chunks.
    """
    combined = "\n\n---\n\n".join(retrieved_chunks)

    response = client.chat.completions.create(
        model="gpt-4o-mini",  # cheap model for this step
        messages=[
            {
                "role": "system",
                "content": (
                    "Extract only the sentences or short passages from the provided text "
                    "that are directly relevant to answering the query. "
                    "Remove everything else. Preserve the meaning of what you keep. "
                    "Do not add anything that is not in the source text."
                ),
            },
            {
                "role": "user",
                "content": f"Query: {query}\n\nText:\n{combined}",
            },
        ],
        max_tokens=800,
    )

    return response.choices[0].message.content


# Usage in your RAG pipeline:
# compressed = compress_context(user_query, retrieved_chunks)
# final_response = expensive_model_call(user_query, compressed)

This adds a small cost for the compression step, but the reduction in context sent to the main model more than covers it — typically 3–4x reduction in RAG context length.

Model routing: matching model to task

Not every LLM call needs the most capable model you have access to. We maintain a simple routing layer that assigns each call type to the cheapest model that handles it reliably.

The categories we use:

Classification and intent detection — small models perform as well as large ones here, and often better when the label set is well-defined. We use gpt-4o-mini or claude-haiku for these.
Extraction from structured documents — similar story. If you know what you are looking for and the documents are reasonably well-formatted, small models are fine.
Complex reasoning, nuanced generation, multi-step planning — this is where the large models earn their cost. Do not route these away.
Summarisation — depends heavily on the required quality. For internal summaries (digests, admin views), cheaper models are fine. For customer-facing summaries that represent your product, use the better model.

from enum import Enum
from dataclasses import dataclass


class TaskComplexity(Enum):
    SIMPLE = "simple"       # classification, extraction, yes/no
    MODERATE = "moderate"   # summarisation, short generation
    COMPLEX = "complex"     # reasoning, planning, nuanced generation


@dataclass
class ModelConfig:
    model: str
    max_tokens: int


ROUTING_TABLE: dict[TaskComplexity, ModelConfig] = {
    TaskComplexity.SIMPLE: ModelConfig(model="gpt-4o-mini", max_tokens=256),
    TaskComplexity.MODERATE: ModelConfig(model="gpt-4o-mini", max_tokens=1024),
    TaskComplexity.COMPLEX: ModelConfig(model="gpt-4o", max_tokens=2048),
}


def get_model_config(complexity: TaskComplexity) -> ModelConfig:
    return ROUTING_TABLE[complexity]


# Example: classifying support tickets
def classify_support_ticket(ticket_text: str) -> str:
    config = get_model_config(TaskComplexity.SIMPLE)

    response = client.chat.completions.create(
        model=config.model,
        messages=[
            {
                "role": "system",
                "content": (
                    "Classify the support ticket into exactly one of: "
                    "billing, technical, account, feature_request, other. "
                    "Reply with only the category name."
                ),
            },
            {"role": "user", "content": ticket_text},
        ],
        max_tokens=config.max_tokens,
    )

    return response.choices[0].message.content.strip()

The key discipline here is to benchmark each task type with both model tiers before committing to the cheaper option. "It seems fine" is not a good enough standard. We run each call type through 50–100 real production examples, grade the outputs, and only route to the cheaper model if quality is within an acceptable margin.

What we tried that did not work

Aggressive output length constraints. We tried setting low max_tokens on generation tasks to reduce output cost. It saved a small amount but made the outputs worse — models truncate in ways that break coherence. Output token cost is usually not the problem; do not sacrifice quality here.

Batching requests. The batch API reduces cost by ~50% on some providers but introduces latency of minutes to hours. For anything user-facing, the tradeoff is not worth it. It works for offline processing jobs where latency does not matter.

Switching providers entirely based on benchmark performance. We spent time evaluating alternative providers that were cheaper per token. For some tasks they were fine. For others, the quality drop was meaningful and affected the product. The benchmarks do not tell you how a model performs on your specific task with your specific data — only testing on your own workload does.

The honest summary

Sixty percent cost reduction sounds dramatic. In practice it came from three things applied together: semantic caching (biggest impact), smarter model routing (second biggest), and prompt/context compression (smaller but meaningful). None of these required rearchitecting anything fundamental.

The prerequisite for all of it was instrumentation. You cannot optimise what you cannot measure. Log every call, log token counts by call type, and look at the data before deciding where to focus. The calls that feel expensive are often not the ones that are actually expensive.

The other thing worth saying: do not optimise prematurely. If your LLM spend is $300 a month and growing slowly, the engineering time to implement semantic caching is not worth it yet. Do it when the numbers justify it — and make sure the instrumentation is already in place so you know when that point arrives.

Lycore builds production AI systems for businesses — RAG pipelines, AI agents, LLM integrations, and custom AI applications built for scale and reliability. Get in touch if you want to talk through your use case.

Multi-Agent Systems in Production: When One Agent Isn't Enough and How We Coordinate Them

Lycore Development — Sun, 28 Jun 2026 15:19:00 +0000

We built our first "multi-agent system" by accident. What started as a single agent that could research a topic, draft a report, check it against source data, and send a summary email had grown into a 2,000-token system prompt and a function list so long that the model kept forgetting tools existed. It wasn't a system — it was a monolith pretending to be intelligent.

Breaking it apart into coordinated agents fixed most of the problems. It also introduced a new category of problems we hadn't thought about. Here's what we actually learned.

When One Agent Is Enough (and When It Isn't)

The temptation to add more agents is real, but the overhead isn't free. Every agent boundary you add is a place where context can get lost, latency increases, and errors compound.

One agent is the right call when:

The task fits in a single LLM context window without crowding
The steps are sequential and each depends heavily on the prior output
You need tight reasoning across all the information (summarising a document, for example)

You need multiple agents when:

A single agent's context window is being maxed out with tool definitions, history, or data
Different steps require genuinely different "personas" or instruction sets (research vs. writing vs. fact-checking)
Steps can run in parallel and the latency saving matters
You want to isolate failure — if the data extraction agent fails, the report-writing agent shouldn't be affected

The key question we ask: Is this one job or a pipeline of jobs? If you'd describe it to a human as "first do X, then Y takes that and does Z", you probably have a pipeline, not a single task.

The Three Patterns We Actually Use

1. Supervisor-Worker

A thin orchestrator agent decides what needs doing, dispatches to specialised worker agents, and stitches the results together. The workers are narrow — they do one thing and don't need to know about the rest of the workflow.

This is our most common pattern. The supervisor's system prompt stays small because it's routing, not reasoning. The workers' prompts can be highly optimised for their specific job.

2. Sequential Pipeline

Each agent's output is the next agent's input. No orchestrator — just a chain. We use this for document processing: extract → chunk → summarise → classify. Each step is independent enough that we can swap out or retrain one without touching the others.

3. Event-Driven Agents

Agents subscribe to events rather than being called directly. An intake agent processes a new customer request and emits an event; a triage agent picks it up, classifies it, and emits another; a response agent drafts the reply. We use this with Celery and Redis when the steps can happen asynchronously and we don't need the full chain to complete before responding to the user.

The Orchestrator in Django + Celery

Here's a simplified version of how we implement the supervisor pattern. The orchestrator Celery task manages the workflow; individual agent tasks do the actual LLM calls.

# tasks/orchestrator.py
from celery import chain, chord
from .agents import extract_data_task, analyse_data_task, draft_report_task

@app.task(bind=True, max_retries=3)
def run_report_pipeline(self, document_id: int, user_id: int):
    """
    Supervisor: extract → analyse → draft, with error isolation at each step.
    """
    try:
        # Build the pipeline as a Celery chain
        pipeline = chain(
            extract_data_task.s(document_id),
            analyse_data_task.s(user_id=user_id),
            draft_report_task.s(user_id=user_id),
        )
        result = pipeline.apply_async()
        return {"pipeline_id": result.id, "status": "started"}

    except Exception as exc:
        # Retry with exponential backoff before giving up
        raise self.retry(exc=exc, countdown=2 ** self.request.retries)

Each agent task is responsible for its own LLM call and its own error handling. The orchestrator doesn't need to know what model each agent uses, or whether agent two calls a tool — it just cares about the shape of the data passing between steps.

Passing State Between Agents Without Losing Your Mind

The naïve approach is to pass the full output of each agent directly into the next. This breaks down fast: LLM outputs are verbose, and feeding 3,000 tokens of analysis into a drafter that only needs 5 key facts wastes tokens and degrades quality.

We use a structured intermediate format — a plain Python dataclass or Pydantic model — as the contract between agents. Each agent's output is validated against this schema before it's passed downstream.

from pydantic import BaseModel
from typing import Optional

class ExtractionResult(BaseModel):
    document_id: int
    key_facts: list[str]          # Max 10 bullet points
    raw_data_summary: str          # Under 500 chars
    confidence_score: float        # 0–1
    extraction_warnings: list[str] # Anything the agent flagged

class AnalysisResult(BaseModel):
    document_id: int
    findings: list[str]
    risk_flags: list[str]
    recommended_actions: list[str]
    analysis_notes: Optional[str] = None

# In the extraction agent task:
@app.task
def extract_data_task(document_id: int) -> dict:
    raw_output = call_llm(
        system="You are a data extraction specialist...",
        user=get_document_text(document_id),
        response_format=ExtractionResult,  # Structured output enforced
    )
    result = ExtractionResult.model_validate(raw_output)
    return result.model_dump()  # Celery serialises as dict

Enforcing the schema at the boundary means your analysis agent never has to guess what the extraction agent gave it. When something breaks, the error is at the boundary where it belongs, not buried three steps later.

Handling Failures in a Multi-Agent Chain

The hardest part of multi-agent systems is failure handling. In a monolithic agent, one failure terminates one task. In a pipeline, a failure in step two means you've wasted step one and need to decide whether to retry from the start or from step two.

Our approach:

Checkpoint results to the database after each step, not just at the end. If step three fails, we can replay from step two's saved output.
Each agent retries independently with a backoff before propagating failure. Most LLM failures are transient.
The orchestrator tracks pipeline state — we have a PipelineRun model with status fields for each step. This lets us resume partial pipelines and gives us visibility into where things are breaking.

# models.py
class PipelineRun(models.Model):
    document = models.ForeignKey(Document, on_delete=models.CASCADE)
    status = models.CharField(max_length=20, default='pending')

    # Checkpointed results per step
    extraction_result = models.JSONField(null=True)
    analysis_result = models.JSONField(null=True)
    draft_result = models.JSONField(null=True)

    # Step-level status
    extraction_status = models.CharField(max_length=20, default='pending')
    analysis_status = models.CharField(max_length=20, default='pending')
    draft_status = models.CharField(max_length=20, default='pending')

    error_detail = models.TextField(blank=True)
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

This makes debugging a failed pipeline actually feasible. You open the admin, find the PipelineRun, see which step failed, and read the error. Without this, you're parsing Celery logs hoping something tells you what happened.

The Honest Summary

Multi-agent architectures solve real problems — context overflow, specialisation, parallelism, and failure isolation. But they introduce coordination overhead that a single agent doesn't have. You're trading simplicity for scalability and resilience.

The things this doesn't solve: it won't fix a poorly designed system prompt on an individual agent, it won't save you if your task decomposition is wrong, and it adds latency. Every agent boundary is a round-trip to an LLM.

Start with one agent. Add a second when you have a clear reason — not because it sounds more impressive. The moment you're debugging why agent three hallucinated because agent two gave it a vague extraction result, you'll appreciate the value of simple.

We run multi-agent pipelines in production for document processing, automated research workflows, and customer triage. They work well, but every one of them started life as a single agent that we only split apart when we had a concrete reason.

Lycore builds production AI systems for businesses — we design and implement multi-agent pipelines, RAG systems, and LLM integrations that hold up in production. Get in touch if you want to talk through your use case.

Structured Outputs: How We Stopped Parsing LLM Responses by Hand

Lycore Development — Sat, 27 Jun 2026 10:19:18 +0000

Every team we talk to has a version of the same story. They built an LLM integration that works well in testing. Then, three weeks into production, something comes back slightly different — the model wraps the JSON in a code block, or uses "status": "Completed" instead of "status": "complete", or includes an extra key that breaks the downstream parser. The whole pipeline falls over.

This post is about how we handle that problem — specifically, how we use structured outputs to get reliable, typed data from LLMs in production Django applications, and where the approach still has limits.

The problem with parsing free-text LLM responses

When you ask an LLM to "return JSON", it usually does. Until it doesn't.

The failure modes are predictable once you've seen them enough times:

The model wraps the output in a markdown code fence (json ...)
Field names drift slightly (customer_id vs customerId vs customer id)
Optional fields are sometimes present, sometimes absent, with no consistency
The model adds a conversational sentence before or after the JSON
Numeric fields come back as strings in edge cases

None of this is surprising — the model is a text predictor, not a JSON serialiser. Treating its output as reliable structured data requires you to either enforce structure at generation time, or write defensive parsing code that handles every variant. The second path is a maintenance problem that compounds over time.

Structured outputs enforce schema at generation time

The cleaner approach is to constrain what the model can generate. OpenAI's structured outputs feature (available since late 2024) lets you pass a JSON schema to the API, and the model is guaranteed to return output that conforms to it. No code fences, no stray fields, no type mismatches.

We define our schemas with Pydantic and pass them directly to the API:

from pydantic import BaseModel
from openai import OpenAI
from typing import Literal

client = OpenAI()


class ExtractionResult(BaseModel):
    company_name: str
    industry: str
    annual_revenue_usd: int | None
    employee_count: int | None
    confidence: Literal["high", "medium", "low"]
    notes: str


def extract_company_info(raw_text: str) -> ExtractionResult:
    response = client.beta.chat.completions.parse(
        model="gpt-4o-2024-08-06",
        messages=[
            {
                "role": "system",
                "content": (
                    "Extract structured company information from the provided text. "
                    "Use null for fields you cannot determine with reasonable confidence."
                ),
            },
            {"role": "user", "content": raw_text},
        ],
        response_format=ExtractionResult,
    )
    return response.choices[0].message.parsed

The return value is a proper Pydantic model instance. You can access result.company_name directly, pass it to a Django serializer, store it in a JSONField — it is typed data, not a string you have to parse.

What this looks like in a real Django pipeline

We use this pattern in a document processing pipeline where we extract key fields from uploaded contracts and business documents before routing them for human review.

# models.py
from django.db import models


class Document(models.Model):
    STATUS_CHOICES = [
        ("pending", "Pending"),
        ("processing", "Processing"),
        ("extracted", "Extracted"),
        ("failed", "Failed"),
        ("needs_review", "Needs Review"),
    ]

    file = models.FileField(upload_to="documents/")
    raw_text = models.TextField(blank=True)
    extracted_data = models.JSONField(null=True, blank=True)
    extraction_confidence = models.CharField(max_length=10, blank=True)
    status = models.CharField(max_length=20, choices=STATUS_CHOICES, default="pending")
    created_at = models.DateTimeField(auto_now_add=True)


# tasks.py (Celery)
from celery import shared_task
from openai import OpenAI
from pydantic import BaseModel, ValidationError
from typing import Literal
import logging

logger = logging.getLogger(__name__)
client = OpenAI()


class ContractExtraction(BaseModel):
    counterparty_name: str
    contract_value_usd: int | None
    start_date: str | None  # ISO 8601
    end_date: str | None
    auto_renewal: bool
    governing_law: str | None
    confidence: Literal["high", "medium", "low"]


@shared_task
def extract_document_fields(document_id: int):
    from .models import Document

    doc = Document.objects.get(id=document_id)
    doc.status = "processing"
    doc.save(update_fields=["status"])

    try:
        response = client.beta.chat.completions.parse(
            model="gpt-4o-2024-08-06",
            messages=[
                {
                    "role": "system",
                    "content": (
                        "Extract key fields from this contract. "
                        "Use null for fields not present or unclear. "
                        "Set confidence to 'low' if you are uncertain about any critical field."
                    ),
                },
                {"role": "user", "content": doc.raw_text[:8000]},  # Stay within context
            ],
            response_format=ContractExtraction,
        )

        result = response.choices[0].message.parsed

        doc.extracted_data = result.model_dump()
        doc.extraction_confidence = result.confidence
        doc.status = "needs_review" if result.confidence == "low" else "extracted"

    except Exception as e:
        logger.error(f"Extraction failed for document {document_id}: {e}")
        doc.status = "failed"

    doc.save()

The key decision here: low-confidence extractions automatically route to human review. The confidence field is part of the schema — we instruct the model to self-report uncertainty, and we act on it. This is the same principle as our agent designs: the human review path is first-class, not a fallback.

Handling refusals

The one case structured outputs cannot prevent is a model refusal. If the model decides the input violates its content policy, response.choices[0].message.parsed will be None and response.choices[0].message.refusal will contain the refusal message.

This needs explicit handling:

message = response.choices[0].message

if message.refusal:
    logger.warning(f"Model refused extraction for document {document_id}: {message.refusal}")
    doc.status = "needs_review"
    doc.save(update_fields=["status"])
    return

result = message.parsed

In practice, refusals are rare for document extraction tasks. They are more common when you are doing classification or analysis on content that might be flagged — customer support tickets, forum posts, unmoderated user content. If your pipeline processes that kind of input, test refusal handling early.

Anthropic's equivalent: tool use

If you are using Anthropic's Claude models (which we also use for some tasks), the equivalent mechanism is tool use. You define a tool with a JSON schema, instruct the model to always call it, and get structured output through the tool call rather than the message content.

import anthropic
import json

client = anthropic.Anthropic()

extraction_tool = {
    "name": "extract_contract_fields",
    "description": "Extract structured fields from the contract text.",
    "input_schema": {
        "type": "object",
        "properties": {
            "counterparty_name": {"type": "string"},
            "contract_value_usd": {"type": ["integer", "null"]},
            "start_date": {"type": ["string", "null"]},
            "end_date": {"type": ["string", "null"]},
            "auto_renewal": {"type": "boolean"},
            "confidence": {"type": "string", "enum": ["high", "medium", "low"]},
        },
        "required": ["counterparty_name", "auto_renewal", "confidence"],
    },
}


def extract_with_claude(raw_text: str) -> dict:
    response = client.messages.create(
        model="claude-opus-4-5",
        max_tokens=1024,
        tools=[extraction_tool],
        tool_choice={"type": "tool", "name": "extract_contract_fields"},
        messages=[
            {"role": "user", "content": f"Extract fields from this contract:\n\n{raw_text}"}
        ],
    )

    tool_use_block = next(b for b in response.content if b.type == "tool_use")
    return tool_use_block.input  # Already a dict, schema-validated

The tool_choice parameter forces the model to always call the specified tool rather than choosing to respond in prose. Without it, the model might sometimes call the tool and sometimes answer in text — not useful in a production pipeline.

What structured outputs do not solve

A few things worth being clear about:

They do not fix bad prompts. If your system prompt is vague about what a field should contain, you will get consistent structure but inconsistent semantics. confidence: "high" means whatever the model inferred it means, not whatever you intended. Schema design and prompt design go together.

They do not prevent hallucination. The model can still make up a contract value or misattribute a date. You are getting reliably shaped data — its accuracy still depends on the model's reasoning and the quality of the source text. For high-stakes fields, add a verification step that cross-checks extracted values against source text.

They add latency. Structured output generation with constrained decoding is slightly slower than unconstrained generation. For real-time user-facing features, measure this before committing to the pattern. For background processing pipelines, it generally does not matter.

The honest summary

Structured outputs are not exotic — they are just the right default when you need typed data from an LLM. Free-text parsing is a trap that costs you maintenance time and production incidents over the long run.

If you are building an LLM integration that outputs data to a database, an API, or another system: define a Pydantic schema, use response_format, handle refusals, and route low-confidence results to human review. That is the pattern. It is not complicated once you have seen it, but it makes a meaningful difference in how reliably the system runs.

Lycore builds production AI systems for businesses — document intelligence, agents, RAG pipelines, and custom LLM integrations on Django, React, Flutter, and .NET. Get in touch if you want to talk through your use case.

Grok vs Gemini: A Developer's Honest Comparison for Real-World Use Cases

Lycore Development — Wed, 03 Jun 2026 00:55:00 +0000

The Model Comparison Problem

Most AI model comparisons are useless for developers making real decisions.

They benchmark on academic datasets that don't reflect production workloads. They test frontier capabilities that matter for 5% of use cases. They ignore latency, cost, rate limits, and API reliability — which are the things that actually determine whether a model works in your application.

This comparison is different. It's focused on what matters when you're building something: how Grok and Gemini perform on the types of tasks developers actually encounter, what each model's API experience is like, and where the genuine tradeoffs lie.

I'm deliberately not including benchmark scores. If you want MMLU numbers, there are plenty of leaderboards for that. This is about production utility.

What Each Model Actually Is

Grok (xAI)

Grok is xAI's model family. The current production models are Grok-3 and Grok-3 Mini, with Grok-3 being the flagship. Grok has a large context window (128K tokens standard, with extended context available), real-time access to X (Twitter) data as a differentiating feature, and strong performance on reasoning-heavy tasks.

The xAI API follows a familiar REST pattern and is broadly compatible with OpenAI SDK conventions, which makes migration straightforward.

Grok's notable characteristics:

Strong at structured reasoning and multi-step problem decomposition
Real-time web access via the API (useful for tasks needing current information)
Relatively generous rate limits compared to some competitors
Less restrictive on certain content categories than some other models

Gemini (Google DeepMind)

Gemini is Google's model family, currently anchored by Gemini 1.5 Pro and Gemini 2.0 Flash. The defining feature of Gemini is its context window — Gemini 1.5 Pro supports up to 1 million tokens in production, which is genuinely useful for certain document-heavy use cases.

Gemini also has the tightest integration with Google's ecosystem (Workspace, Cloud, Search), which matters if you're building in that stack.

Gemini's notable characteristics:

Industry-leading context window (1M tokens for 1.5 Pro)
Strong multimodal capability (video, audio, images, text in the same context)
Native Google ecosystem integration
Gemini 2.0 Flash is very fast and cheap — competitive with smaller models from other providers

Head-to-Head: Task-by-Task

Code Generation and Review

Both models write competent code. The practical differences:

Grok tends to produce more concise implementations, often hitting the right solution without over-engineering. It handles edge cases well when they're described explicitly in the prompt.

Gemini (particularly 1.5 Pro) excels when you can give it a large codebase as context — its million-token window means you can drop in entire repositories and ask questions about them. For "explain this code" or "find the bug in this file" tasks on large codebases, nothing else matches it.

import anthropic
from google import generativeai as genai
import os

# Grok via xAI API (OpenAI-compatible)
from openai import OpenAI

def code_review_grok(code: str, language: str) -> str:
    client = OpenAI(
        api_key=os.environ["XAI_API_KEY"],
        base_url="https://api.x.ai/v1"
    )
    response = client.chat.completions.create(
        model="grok-3",
        messages=[
            {
                "role": "system",
                "content": "You are a senior software engineer doing a thorough code review. Focus on bugs, security issues, performance problems, and maintainability."
            },
            {
                "role": "user",
                "content": f"Review this {language} code:\n\n```
{% endraw %}
{language}\n{code}\n
{% raw %}
```"
            }
        ],
        temperature=0.1
    )
    return response.choices[0].message.content

def code_review_gemini(code: str, language: str, full_codebase: str = None) -> str:
    genai.configure(api_key=os.environ["GOOGLE_API_KEY"])
    model = genai.GenerativeModel("gemini-1.5-pro")

    context = ""
    if full_codebase:
        # Gemini's killer feature: pass the entire codebase for context
        context = f"\n\nFull codebase context:\n{full_codebase}"

    prompt = f"""Review this {language} code for bugs, security issues, and maintainability problems.

Code to review:

{language}
{code}

```{context}"""

response = model.generate_content(prompt)
return response.text

Verdict: Use Gemini 1.5 Pro when you have large codebase context to include.

Use Grok for standalone code review tasks — slightly faster, more concise output.




**Verdict for code tasks**: Gemini 1.5 Pro for large-context code analysis. Grok 3 for standard code generation and review. Gemini 2.0 Flash for high-volume, lower-complexity coding assistance where cost matters.

---

### Structured Data Extraction

Both models handle JSON output well when prompted correctly. Grok is slightly more consistent at following strict schemas without additional enforcement.



```python
import json
from openai import OpenAI
import google.generativeai as genai

EXTRACTION_SCHEMA = {
    "company_name": "string",
    "funding_round": "string (seed/series-a/series-b/etc)",
    "amount_usd": "number or null",
    "investors": ["list of investor names"],
    "announcement_date": "YYYY-MM-DD or null"
}

def extract_funding_grok(article_text: str) -> dict:
    client = OpenAI(api_key=os.environ["XAI_API_KEY"], base_url="https://api.x.ai/v1")

    response = client.chat.completions.create(
        model="grok-3",
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": f"Extract funding information. Return ONLY valid JSON matching: {json.dumps(EXTRACTION_SCHEMA)}"},
            {"role": "user", "content": article_text}
        ],
        temperature=0
    )
    return json.loads(response.choices[0].message.content)

def extract_funding_gemini(article_text: str) -> dict:
    genai.configure(api_key=os.environ["GOOGLE_API_KEY"])
    model = genai.GenerativeModel(
        "gemini-2.0-flash",
        generation_config={"response_mime_type": "application/json"}
    )

    prompt = f"""Extract funding information from this article and return JSON matching exactly:
{json.dumps(EXTRACTION_SCHEMA, indent=2)}

Article:
{article_text}"""

    response = model.generate_content(prompt)
    return json.loads(response.text)

# Gemini 2.0 Flash is significantly cheaper here and performs nearly identically.
# For high-volume extraction pipelines, Flash wins on cost.

Verdict for structured extraction: Gemini 2.0 Flash at scale (cost efficiency is significant). Grok 3 when schema adherence is critical and you want belt-and-suspenders reliability.

Long Document Analysis

This is Gemini's clearest win. The 1-million-token context window is not a gimmick — for legal document review, large codebase analysis, processing lengthy research reports, or summarising books, it changes what's possible.

Grok's 128K context handles most practical documents comfortably, but there are genuine use cases where Gemini 1.5 Pro's context advantage matters.

def analyse_long_document_gemini(document_text: str, questions: list[str]) -> dict:
    """
    Gemini 1.5 Pro can handle documents up to ~750,000 words.
    Useful for: legal contracts, technical specifications, large codebases,
    research compilations, lengthy transcripts.
    """
    genai.configure(api_key=os.environ["GOOGLE_API_KEY"])
    model = genai.GenerativeModel("gemini-1.5-pro")

    prompt = f"""Analyse this document and answer the following questions. 
For each answer, cite the relevant section of the document.

Document:
{document_text}

Questions:
{chr(10).join(f"{i+1}. {q}" for i, q in enumerate(questions))}

Return answers as JSON: {{"answers": [{{"question": "...", "answer": "...", "citation": "..."}}]}}"""

    response = model.generate_content(prompt)
    return json.loads(response.text)

Verdict for long documents: Gemini 1.5 Pro, not close. The context window advantage is real and significant.

Real-Time and Current Information

Grok's integration with real-time X data is a genuine differentiator for use cases that need current information. For social sentiment analysis, tracking trending topics, or getting context on recent events, this is built in rather than requiring a separate search integration.

def get_current_context_grok(topic: str) -> str:
    """Grok can access real-time X data for current context."""
    client = OpenAI(api_key=os.environ["XAI_API_KEY"], base_url="https://api.x.ai/v1")

    response = client.chat.completions.create(
        model="grok-3",
        messages=[{
            "role": "user",
            "content": f"What are the latest developments and current sentiment around: {topic}? Include recent context from the past 24-48 hours."
        }]
    )
    return response.choices[0].message.content

# Gemini has web search via Google Search grounding, but the integration
# is less seamless than Grok's X data access.

Verdict for real-time info: Grok for social/market sentiment and current events. Gemini with Search grounding for general web information.

API Experience and Ecosystem

Factor	Grok (xAI)	Gemini (Google)
SDK quality	Good (OpenAI-compatible)	Good (native SDK + OpenAI-compatible)
Rate limits	Generous for dev tier	Tiered; Flash very generous
Pricing	Competitive	Flash is among cheapest available
Reliability	Good, improving	Very good (Google infrastructure)
Google ecosystem	None	Native (Workspace, Cloud, Search)
Streaming	Yes	Yes
Function calling	Yes	Yes

When to Choose Which

Choose Grok when:

You need real-time X/social data in your application
You want OpenAI SDK compatibility with minimal migration effort
Your task involves current events or recent information
You want strong reasoning without the full cost of frontier models

Choose Gemini 1.5 Pro when:

Your use case involves very large documents or codebases (>100K tokens)
You need multimodal (video, audio, image + text) in the same context
You're building in Google Cloud or Workspace
Long-context retrieval accuracy is the primary requirement

Choose Gemini 2.0 Flash when:

Cost efficiency is critical and you're running high volume
Latency matters and you need fast response times
The task doesn't require frontier-model reasoning depth

The honest answer for most use cases: the capability difference between these models and the other frontier options (Claude, GPT-4) is smaller than the marketing suggests. Architectural decisions — prompt design, caching, context management, output validation — matter more than model choice for most production applications. Choose the model whose API pricing, rate limits, and ecosystem integration fit your stack, and focus your engineering energy on building the application layer well.

For teams evaluating their AI stack and making model selection decisions, Lycore has written a detailed comparison covering the full landscape of available models — including Claude and GPT-4 — with a focus on production decision-making rather than benchmark scores.

What's your experience been with these models in production? I'm particularly curious about anyone who's migrated between providers — what were the friction points?

AI Is Not Killing Developer Jobs — But It Is Killing Certain Developer Habits

Lycore Development — Thu, 28 May 2026 23:33:00 +0000

The Headline vs. The Reality

"AI is replacing developers." It's everywhere. Breathless predictions about software engineers being the first white-collar profession to be automated away. CEOs citing AI as justification for hiring freezes. Boot camps quietly pivoting their messaging.

The data doesn't support the headline. But the data does show something real — and developers who dismiss the AI-replacement narrative as pure hype are making a different kind of mistake.

This post is my honest read of what's actually happening in the developer job market, what AI tools are genuinely changing about how software gets built, and what that means for how you should be developing your skills and career.

What the Data Actually Shows

Tech layoffs in 2024-2026 have been significant. But when you look at the reasons cited in earnings calls and internal memos, the picture is complicated:

Post-pandemic overhiring correction (the dominant factor at most major tech companies)
Rising interest rates changing the economics of growth-at-all-costs
Consolidation in specific sectors (crypto, ad tech, social media)
Genuine AI-driven productivity improvements enabling smaller teams

The last factor is real, but it's a smaller driver than the narrative suggests. Companies reducing headcount primarily because of overhiring correction are attributing those decisions to AI because it sounds strategic rather than reactive.

What's also real: entry-level developer hiring has slowed meaningfully at large companies. The reason given internally at many is that AI coding tools allow senior developers to handle more work. Whether this is true in practice or rationalization is genuinely unclear — productivity data from AI coding tool deployments is inconsistently reported and often self-serving.

The honest assessment: AI has made it easier to build software with smaller teams (the Stack Overflow Developer Survey 2024 found 76% of developers are using or planning to use AI tools). This changes the hiring math for certain roles, particularly roles that were primarily executing well-defined specifications. It has not changed the scarcity of developers who can design systems, make architectural decisions, and work effectively in ambiguous problem spaces.

What AI Tools Are Actually Replacing

Let's be specific about what AI coding tools do well:

Boilerplate and scaffolding generation

Setting up a new Django project, generating CRUD API endpoints, writing Pytest fixtures, creating database migration scripts — AI does this competently and faster than most developers. Time previously spent on this category of work is genuinely compressible.

# The kind of thing AI generates well — a complete, working FastAPI endpoint
# with validation, error handling, and type hints. Previously took 20 minutes
# to write carefully. Now takes 2 minutes to prompt and review.

from fastapi import APIRouter, HTTPException, Depends
from pydantic import BaseModel, EmailStr
from sqlalchemy.orm import Session
from typing import Optional
import uuid

router = APIRouter(prefix="/api/v1/users", tags=["users"])

class UserCreate(BaseModel):
    email: EmailStr
    full_name: str
    role: str = "member"

class UserResponse(BaseModel):
    id: str
    email: str
    full_name: str
    role: str
    created_at: str

    class Config:
        from_attributes = True

@router.post("/", response_model=UserResponse, status_code=201)
async def create_user(user_data: UserCreate, db: Session = Depends(get_db)):
    existing = db.query(User).filter(User.email == user_data.email).first()
    if existing:
        raise HTTPException(status_code=409, detail="Email already registered")

    user = User(
        id=str(uuid.uuid4()),
        email=user_data.email,
        full_name=user_data.full_name,
        role=user_data.role
    )
    db.add(user)
    db.commit()
    db.refresh(user)
    return user

@router.get("/{user_id}", response_model=UserResponse)
async def get_user(user_id: str, db: Session = Depends(get_db)):
    user = db.query(User).filter(User.id == user_id).first()
    if not user:
        raise HTTPException(status_code=404, detail="User not found")
    return user

Test generation for known patterns

Given a function, AI can generate unit tests covering common cases and obvious edge cases. It misses subtle domain-specific edge cases and doesn't understand business logic the way a developer who wrote the original code does — but for coverage of straightforward paths, it's useful.

Documentation drafting

Docstrings, README sections, API documentation, inline comments explaining non-obvious code. AI produces competent first drafts of all of these. They require review and editing, but the blank page problem is solved.

Debugging assistance

Explaining error messages, suggesting likely causes of bugs, recommending debugging strategies. This is genuinely useful for junior developers and for debugging in unfamiliar codebases or languages.

What AI Tools Are Not Replacing

System design and architecture

Deciding how to structure a system — what the service boundaries are, what data model fits the domain, how to handle concurrency, when to use eventual consistency — requires understanding the business context, the team's capabilities, the scaling requirements, and dozens of tradeoffs that aren't captured in any prompt.

AI can suggest patterns. It cannot make the judgment calls that require understanding context beyond what fits in a context window.

Debugging production systems

Production bugs in complex systems are not well-defined problems. They involve incomplete information, distributed systems interactions, race conditions that appear intermittently, and emergent behaviours that weren't anticipated in design. The debugging process is fundamentally about forming and testing hypotheses with incomplete data. AI assists but does not lead.

Technical leadership

Translating business requirements into technical approaches, managing technical debt strategically, making build vs buy decisions, identifying risks early, communicating complexity to non-technical stakeholders — none of this is close to being automated.

Domain expertise

A developer who deeply understands financial regulation, medical device software requirements, aerospace safety standards, or any other specialised domain cannot be replaced by a general-purpose coding assistant. The domain knowledge is the differentiator.

The Habits That Are Actually at Risk

Here's where the real disruption is — not in developer headcounts, but in which developer habits and skill areas are becoming less valuable:

Memorising syntax and API signatures

If you built a reputation on knowing the exact syntax for every Python built-in or the specific parameters of every React hook, that's less valuable now. AI handles this better than most humans. The habit of reaching for documentation for every unfamiliar API call is being replaced by prompting.

What to do: Invest in understanding fundamentals rather than memorising specifics. Know why things work, not just how to type them.

Writing the same boilerplate patterns repeatedly

The developer who was valuable because they could quickly scaffold a standard CRUD service or set up a standard authentication flow is in a more competitive position. AI does this well.

What to do: Move up the value chain. Be the person who decides what to scaffold and whether the standard pattern is right for this context — not the one executing the scaffolding.

Gatekeeping knowledge

"I know how to do X and you don't" is a weaker moat than it used to be. AI has democratised access to a lot of technical knowledge that was previously held by specialists.

What to do: Build moats that AI can't replicate — deep domain expertise, strong working relationships, a track record of shipping reliably, the ability to navigate ambiguous requirements.

Avoiding unfamiliar technology

"I don't know Rust" or "I've never used Kafka" used to be valid reasons to avoid certain work. AI coding assistants make it meaningfully easier to work in unfamiliar languages and systems.

What to do: Use this as an opportunity rather than a threat. Expand your range. The developer who can work effectively across multiple languages and domains is more valuable, not less, when AI handles the syntax lookup.

What Developers Should Actually Be Worried About

Not their jobs, primarily. But there are legitimate concerns:

The entry-level pipeline is getting harder. If senior developers become more productive with AI tools, companies hire fewer juniors. The path from junior to senior has traditionally run through doing a lot of junior work. If there's less junior work, how do people get the experience to become senior? This is a genuine structural problem that the industry hasn't solved.

The middle tier faces real pressure. Developers who are competent but not exceptional — who execute well-defined tasks reliably but don't design systems or lead technical direction — face the most direct productivity comparison with AI tools. This segment has historically been the largest part of the developer workforce. It's under more pressure than the headline replacement narrative suggests, but less than the catastrophists claim.

Skills rot is faster. The half-life of specific technical knowledge is shortening. What was an advanced skill two years ago is table stakes today. The pace of required learning is accelerating, and developers who aren't actively keeping up face steeper obsolescence curves.

The Practical Response

The developers who will be most resilient over the next five years share some characteristics:

They use AI tools fluently but don't depend on them blindly. They can evaluate AI-generated code critically, understand its failure modes, and know when the AI's suggestion is wrong. This requires deep enough understanding that you're supervising the AI rather than deferring to it.

They have genuine domain expertise in at least one area. Fintech, healthcare, security, data infrastructure, distributed systems — something where the domain knowledge takes years to build and AI can assist but not replace.

They work at the problem level, not the code level. The most AI-resistant developer skill is the ability to understand a business problem, identify what technical approach will actually solve it, and communicate that to stakeholders. This is higher-order work that AI assists with but doesn't perform.

They've built reputations for reliable delivery. Trust, track record, and relationships are not AI-compressible. The developer who ships reliably, communicates honestly, and is easy to work with remains valuable regardless of how good AI tools get.

For a deeper look at how this is playing out across specific developer roles and seniority levels, the team at Lycore has written about the changing landscape for software professionals — including which specialisations are seeing the most impact and what the data actually shows about hiring trends.

The Bottom Line

AI is changing software development. It is not eliminating developers. It is eliminating the parts of development that were always more rote than creative — the boilerplate, the scaffold, the documentation draft.

The developers who are struggling are those whose value was concentrated in those rote parts. The developers who are doing well are those who were already working at the layer above — designing, deciding, leading, and building domain expertise.

If you're earlier in your career, the advice is the same as it's always been but more urgent: don't be a human autocomplete. Understand systems. Develop opinions about architecture. Build domain expertise. Learn to communicate technical ideas to non-technical people. Ship things and take responsibility for what you ship.

The tools are getting better. That doesn't make the engineering harder. In many ways it makes the interesting parts more accessible. The question is whether you're building toward the interesting parts or staying comfortable in the parts that are being automated.

How are AI coding tools changing how you actually work day to day? I'm curious whether people are finding genuine productivity gains or mostly incremental improvements — honest answers more valuable than the marketing material on either side.

At Lycore, we build production AI systems that handle real complexity — not just demos. If your team is figuring out how to integrate AI into your development workflow, get in touch.

How to Build a Trading Platform: Architecture, Features, and the Hard Engineering Problems

Lycore Development — Fri, 22 May 2026 01:11:00 +0000

Why Trading Platforms Are Among the Hardest Software to Build

Most software has a generous margin for error. A bug in your e-commerce checkout means a failed transaction — annoying, recoverable. A bug in your trading platform's order matching engine means incorrect executions, real financial losses, and potentially regulatory consequences. The gap between "it works" and "it works correctly under all market conditions" is wider in trading software than almost anywhere else.

I've spent time building and reviewing trading platforms across retail brokerage, institutional execution, and DeFi. This post is a practical engineering guide: the architecture decisions that matter, the features you can't cut corners on, and the failure modes that will bite you if you're not prepared.

This is not financial advice, and building a regulated trading platform requires legal and compliance expertise beyond the scope of any engineering post. What this covers is the engineering substance of the problem.

The Core Components Every Trading Platform Needs

1. Order Management System (OMS)

The OMS is the heart of the platform. It receives orders from users, validates them, routes them for execution, tracks their lifecycle, and reconciles the results. Every other component interacts with it.

Key requirements:

Idempotency: Order submission must be idempotent. Network timeouts are common; if a user retries a submission, you must not create duplicate orders.
State machine correctness: An order has a defined lifecycle (pending → submitted → partially filled → filled, or pending → cancelled, etc.). Transitions must be atomic and auditable.
Audit trail: Every state change, every modification, every cancellation must be logged with timestamp, actor, and reason. This is not optional in any regulated context.

from enum import Enum
from dataclasses import dataclass, field
from datetime import datetime
from typing import Optional
import uuid

class OrderStatus(str, Enum):
    PENDING = "pending"
    SUBMITTED = "submitted"
    PARTIALLY_FILLED = "partially_filled"
    FILLED = "filled"
    CANCELLED = "cancelled"
    REJECTED = "rejected"
    EXPIRED = "expired"

class OrderSide(str, Enum):
    BUY = "buy"
    SELL = "sell"

class OrderType(str, Enum):
    MARKET = "market"
    LIMIT = "limit"
    STOP = "stop"
    STOP_LIMIT = "stop_limit"

@dataclass
class Order:
    user_id: str
    symbol: str
    side: OrderSide
    order_type: OrderType
    quantity: float
    limit_price: Optional[float] = None
    stop_price: Optional[float] = None

    # System-managed fields
    order_id: str = field(default_factory=lambda: str(uuid.uuid4()))
    client_order_id: Optional[str] = None  # Idempotency key from client
    status: OrderStatus = OrderStatus.PENDING
    filled_quantity: float = 0.0
    average_fill_price: Optional[float] = None
    created_at: datetime = field(default_factory=datetime.utcnow)
    updated_at: datetime = field(default_factory=datetime.utcnow)

    def validate(self) -> list[str]:
        """Validate order before submission. Returns list of error messages."""
        errors = []

        if self.quantity <= 0:
            errors.append("Quantity must be positive")

        if self.order_type in (OrderType.LIMIT, OrderType.STOP_LIMIT):
            if self.limit_price is None or self.limit_price <= 0:
                errors.append("Limit price required and must be positive")

        if self.order_type in (OrderType.STOP, OrderType.STOP_LIMIT):
            if self.stop_price is None or self.stop_price <= 0:
                errors.append("Stop price required and must be positive")

        return errors

    def can_transition_to(self, new_status: OrderStatus) -> bool:
        """Enforce valid state machine transitions."""
        valid_transitions = {
            OrderStatus.PENDING: {OrderStatus.SUBMITTED, OrderStatus.REJECTED},
            OrderStatus.SUBMITTED: {
                OrderStatus.PARTIALLY_FILLED, OrderStatus.FILLED,
                OrderStatus.CANCELLED, OrderStatus.EXPIRED
            },
            OrderStatus.PARTIALLY_FILLED: {
                OrderStatus.FILLED, OrderStatus.CANCELLED
            },
        }
        return new_status in valid_transitions.get(self.status, set())


class OrderManagementSystem:

    def __init__(self, db, risk_engine, execution_router, audit_log):
        self.db = db
        self.risk = risk_engine
        self.router = execution_router
        self.audit = audit_log

    def submit_order(self, order: Order) -> dict:
        # Idempotency check
        if order.client_order_id:
            existing = self.db.find_by_client_order_id(order.client_order_id)
            if existing:
                return {"status": "duplicate", "order_id": existing.order_id}

        # Validation
        errors = order.validate()
        if errors:
            return {"status": "rejected", "errors": errors}

        # Pre-trade risk checks
        risk_result = self.risk.check(order)
        if not risk_result.approved:
            order.status = OrderStatus.REJECTED
            self.db.save(order)
            self.audit.log("order_rejected", order, reason=risk_result.reason)
            return {"status": "rejected", "reason": risk_result.reason}

        # Submit
        order.status = OrderStatus.SUBMITTED
        self.db.save(order)
        self.audit.log("order_submitted", order)

        # Route to execution (async in production)
        self.router.route(order)

        return {"status": "submitted", "order_id": order.order_id}

2. Market Data Infrastructure

Your platform needs real-time market data: current prices, order book depth, trade history, and historical data for charts. This is harder than it looks because:

Volume is high: A single liquid equity can generate thousands of price updates per second
Latency matters: Stale prices cause bad user decisions and, in some architectures, bad executions
Data quality matters: Bad ticks (erroneous price prints) need to be filtered

The architecture decision is whether to build your own market data pipeline or use a managed provider. For most platforms, managed providers (Polygon.io, Alpaca, Interactive Brokers data feeds) are the right answer — the engineering investment in a production-grade market data system is substantial and the differentiation is minimal.

When you do need to build your own data handling layer, a time-series database is essential. TimescaleDB (Postgres extension) handles most use cases well without introducing a new operational dependency:

-- TimescaleDB hypertable for OHLCV data
CREATE TABLE ohlcv (
    time        TIMESTAMPTZ NOT NULL,
    symbol      TEXT NOT NULL,
    open        NUMERIC(18, 8) NOT NULL,
    high        NUMERIC(18, 8) NOT NULL,
    low         NUMERIC(18, 8) NOT NULL,
    close       NUMERIC(18, 8) NOT NULL,
    volume      NUMERIC(24, 8) NOT NULL
);

SELECT create_hypertable('ohlcv', 'time');
CREATE INDEX ON ohlcv (symbol, time DESC);

-- Continuous aggregate for 1-hour candles from tick data
CREATE MATERIALIZED VIEW ohlcv_1h
WITH (timescaledb.continuous) AS
SELECT
    time_bucket('1 hour', time) AS bucket,
    symbol,
    first(open, time) AS open,
    max(high) AS high,
    min(low) AS low,
    last(close, time) AS close,
    sum(volume) AS volume
FROM ohlcv
GROUP BY bucket, symbol;

3. Risk Engine

The risk engine sits between order submission and execution. It enforces position limits, buying power constraints, and market risk parameters. It is not optional.

Pre-trade risk checks for a retail platform typically include:

Buying power: Does the user have sufficient funds/margin to cover this order?
Position limits: Would this order exceed maximum allowed position size per symbol?
Order size limits: Is this order unreasonably large (potential fat-finger error)?
Market hours: Is this market currently open for the order type being submitted?
Symbol restrictions: Is this symbol available for trading on this platform?

from dataclasses import dataclass

@dataclass
class RiskCheckResult:
    approved: bool
    reason: Optional[str] = None
    warnings: list = field(default_factory=list)

class PreTradeRiskEngine:

    def __init__(self, account_service, position_service, config):
        self.accounts = account_service
        self.positions = position_service
        self.config = config

    def check(self, order: Order) -> RiskCheckResult:
        account = self.accounts.get(order.user_id)

        # Buying power check
        estimated_cost = self._estimate_order_cost(order)
        if account.available_cash < estimated_cost:
            return RiskCheckResult(
                approved=False,
                reason=f"Insufficient buying power. Required: {estimated_cost:.2f}, Available: {account.available_cash:.2f}"
            )

        # Position limit check
        current_position = self.positions.get(order.user_id, order.symbol)
        new_position = current_position.quantity + (
            order.quantity if order.side == OrderSide.BUY else -order.quantity
        )

        max_position = self.config.get_max_position(order.symbol, account.tier)
        if abs(new_position) > max_position:
            return RiskCheckResult(
                approved=False,
                reason=f"Order would exceed maximum position limit of {max_position} for {order.symbol}"
            )

        # Fat finger check
        if order.quantity > self.config.fat_finger_threshold:
            return RiskCheckResult(
                approved=False,
                reason=f"Order size {order.quantity} exceeds maximum single order size {self.config.fat_finger_threshold}"
            )

        return RiskCheckResult(approved=True)

    def _estimate_order_cost(self, order: Order) -> float:
        if order.order_type == OrderType.LIMIT and order.limit_price:
            return order.quantity * order.limit_price
        # For market orders, use last price with a buffer
        last_price = self.positions.get_last_price(order.symbol)
        return order.quantity * last_price * 1.02  # 2% buffer for market impact

4. Real-Time Portfolio and P&L

Users need to see their current positions, unrealised P&L, and account value in real time. This is a read-heavy workload that benefits from a separate read model updated by the execution feed.

WebSocket connections are the standard for pushing portfolio updates to frontend clients. The architecture: execution fills update a portfolio state store (Redis works well here for latency), and a WebSocket gateway pushes diffs to connected clients.

The Features You Cannot Cut Corners On

Order History and Statements

Every trade must be recorded and retrievable. Users need complete trade history for tax purposes. Regulators need it for compliance purposes. Your operations team needs it for reconciliation.

This means: immutable trade records, complete audit trails, export capabilities (CSV at minimum), and retention policies that meet your regulatory requirements. The retention requirement for financial records in most jurisdictions is 5-7 years.

Account Security

Trading accounts are high-value targets. The security requirements go beyond standard web application security:

MFA mandatory, not optional: SMS, TOTP, or hardware key
Session management: Short session timeouts, concurrent session detection, geographic anomaly alerts
Withdrawal address whitelisting: For crypto platforms, withdrawals only to pre-approved addresses
Transaction monitoring: Flag unusual patterns — unusually large trades, trading at unusual hours, rapid position changes

Reconciliation

End-of-day reconciliation between your internal records and your execution venue records is not optional. Discrepancies exist — execution venues make mistakes, network issues cause message loss, edge cases in your OMS create inconsistencies. Daily automated reconciliation with exception alerting catches these before they compound.

The Infrastructure Reality

A trading platform is not a typical web application. The requirements that differentiate it:

Latency: Order submission to acknowledgement needs to be fast — users notice delays above 200ms, and anything above 1 second creates trust issues. This means database query optimisation, connection pooling, and careful attention to your critical path.

Reliability: Trading platforms need 99.9%+ uptime during market hours. Planned maintenance windows need to be outside market hours. Unplanned outages during high-volatility market sessions are severe reputational events.

Consistency over availability: When you have to choose between availability and consistency (a partition tolerance scenario), trading platforms choose consistency. It is better to reject an order than to create an inconsistent state.

Disaster recovery: You need point-in-time recovery for your trade database, tested regularly. RTO (recovery time objective) and RPO (recovery point objective) need to be defined and designed for before you go live.

For teams building fintech and trading infrastructure, our team at Lycore has hands-on experience with the full stack — from order management systems to real-time market data pipelines to regulatory reporting. The complexity is significant but manageable with the right architecture from the start.

What Most Teams Get Wrong

Starting with the UI: The beautiful trading interface is the last thing to build, not the first. The OMS, risk engine, and execution connectivity need to be solid before the front end matters.

Underestimating reconciliation: Teams consistently underinvest in reconciliation infrastructure and spend months retrofitting it after launch. Build it in from day one.

Ignoring the operational side: A trading platform needs a full operational runbook, clear escalation paths for execution issues, and relationships with your execution venues' technical support teams. You will have incidents. Being prepared for them is the difference between a recoverable situation and a crisis.

Not testing failure modes: Test what happens when your execution venue connection drops mid-order. Test what happens when the market data feed goes stale. Test what happens when your database primary fails over. These scenarios will occur in production.

Building something in the fintech or trading space? I'm happy to discuss architecture in the comments — the specifics vary a lot by asset class, regulatory jurisdiction, and execution model.

The Future of AI in Business: What's Actually Changing and What's Just Hype

Lycore Development — Wed, 20 May 2026 06:00:00 +0000

Separating Signal From Noise in 2026

Every major technology wave produces the same pattern: genuine capability advances, followed by overclaiming, followed by a correction, followed by actual adoption at scale. We went through it with cloud computing, mobile, and big data. We're going through it with AI now.

The challenge for developers and engineering leaders is calibrating correctly. Dismissing AI as hype means missing genuine capability shifts that will change competitive dynamics in your industry. Believing everything means building on foundations that aren't ready, burning engineering time on features users won't adopt, and making technology decisions you'll regret when the dust settles.

This post is an attempt at calibration — a clear-eyed look at what AI is actually changing in business software, what timelines are realistic, and where the current claims outrun the evidence.

What Is Actually Changing (With Evidence)

1. The cost of generating structured content has collapsed

Three years ago, producing a personalised, well-formatted document — a proposal, a report, a contract summary — required significant human time. Today, a well-prompted language model can produce a first draft that requires light editing rather than full authorship.

This is real and it's being adopted. The categories where it's showing clear ROI:

Customer-facing documents: Proposals, quotes, summaries, follow-up emails
Internal documentation: Meeting notes, incident reports, status updates
Code first drafts: Boilerplate, test scaffolding, repetitive CRUD operations
Data interpretation: "Explain what this chart means" at the analyst tier

The productivity gains are real but unevenly distributed. People who work heavily with structured text — writers, analysts, developers — see meaningful productivity improvements. People whose work is primarily relational, physical, or requires deep domain expertise see smaller gains.

2. Search is being replaced by retrieval-augmented generation in knowledge-heavy applications

Enterprise search has always been disappointing. You search a knowledge base and get a ranked list of potentially relevant documents. You then have to read those documents to find the actual answer.

RAG changes the contract: you ask a question in natural language, and you get an answer — ideally with citations so you can verify it. For knowledge-heavy applications (legal, compliance, customer support, internal IT), this is a genuine step function improvement.

The technology is real. The implementation challenge is data quality. RAG systems are only as good as the documents they retrieve from. If your knowledge base is a graveyard of outdated policies and inconsistent formatting, RAG makes it faster to get wrong answers.

3. Autonomous agents are beginning to handle narrow, well-defined workflows

The agent hype cycle peaked around 2024 with claims of fully autonomous software engineers and self-managing businesses. Reality is more modest but genuinely interesting: agents that handle specific, well-scoped workflows with human oversight checkpoints are working in production.

The categories where this is real today:

Data enrichment pipelines: Agents that look up information, cross-reference sources, and populate structured records
Tier-1 support triage: Classification, routing, and initial response — with human escalation paths
Code review assistance: Automated checks for security issues, style consistency, and common bugs
Report generation: Pulling data from multiple sources and producing narrative summaries

The key word in all of these is "narrow." Agents that work are doing one well-defined thing with clear success criteria and bounded failure modes. Agents that fail are trying to do too much in domains that aren't well-specified.

What Is Being Overclaimed

"AI will replace most knowledge workers within 5 years"

This claim collapses when you look at what knowledge work actually consists of. Most knowledge worker time is spent on: relationship management, judgment calls in ambiguous situations, navigating organizational politics, and communicating with stakeholders. AI assists with the documented, text-based portions of this work. It doesn't handle the rest.

The more accurate framing: AI will handle the rote, repetitive, and document-heavy portions of knowledge work, raising the floor for what each worker can produce. This will reduce headcount growth in some functions. It is unlikely to cause mass displacement in the near term.

"You can replace your entire data team with AI"

This one is being sold hard. The reality: AI can accelerate data analysis, surface anomalies, and generate draft interpretations. It cannot replace the domain expertise required to know which questions are worth asking, why a metric moved, or whether a pattern represents a real business signal or a data quality issue.

Data teams that integrate AI tools well become more productive. They are not eliminated.

"Fully autonomous AI coding will end software development"

GitHub Copilot and similar tools are genuinely useful for certain tasks. They write boilerplate well. They autocomplete familiar patterns. They can generate test cases.

What they cannot do: design systems, make architectural tradeoffs, understand business context, manage technical debt across a large codebase, or navigate the gap between what a specification says and what was actually meant. Software development is not primarily about typing code — it's about understanding problems and making decisions. AI assists with the expression layer. The reasoning layer remains human.

The Business Adoption Curve: Where Different Industries Actually Are

Different industries are at different points in genuine AI adoption, and understanding where your industry sits matters for technology decisions.

Early majority (real ROI being measured now):

Financial services: Fraud detection, credit risk, regulatory reporting
Healthcare: Diagnostic imaging assistance, clinical documentation, drug discovery
Legal: Document review, contract analysis, research assistance
Software development: Code assistance, test generation, documentation

Early adopter phase (pilots showing promise, scale unclear):

Manufacturing: Predictive maintenance, quality control
Retail: Demand forecasting, personalisation at scale
Professional services: Proposal generation, project scoping

Still experimental (genuine capability, adoption friction high):

Education: Personalised tutoring, automated grading
Government: Citizen services, policy analysis
Construction: Project planning, safety monitoring

The distinction matters because early majority means you can study competitors' implementations and learn from their mistakes. Early adopter means you're figuring things out yourself. Still experimental means the technology is ahead of the deployment infrastructure.

The Infrastructure Layer That Determines Everything

The thing most business AI discussions miss is the infrastructure question. AI capabilities are advancing fast. The infrastructure required to use those capabilities reliably in production is advancing more slowly.

The gaps that matter most right now:

Evaluation infrastructure: How do you know when your AI system is working correctly? The testing tools for AI systems are immature compared to those for traditional software. Most teams are flying partially blind.

Cost management: AI API costs are unpredictable and can scale non-linearly with usage. Teams that haven't built cost monitoring and circuit breakers into their AI architecture routinely get surprised by bills.

Data governance: Which data can you send to external AI APIs? For regulated industries, this is not a minor compliance checkbox — it's a fundamental constraint on what AI you can use and where.

Change management: AI features change user workflows. The organisational challenge of getting people to use AI tools effectively is often larger than the engineering challenge of building them.

What This Means for Engineering Decisions Today

If you're making technology decisions with a 2-3 year horizon, the framework we use:

Build now, with confidence:

RAG pipelines for knowledge-heavy applications
LLM-assisted content generation with human review
Narrow workflow automation with defined scope and human oversight
AI-assisted code review and testing

Build now, but architect for change:

AI-powered search and recommendation systems (models and providers will change)
Customer-facing AI features (user expectations are shifting fast)
Anything using frontier model APIs (pricing and capability are moving targets)

Wait for the infrastructure to mature:

Fully autonomous agents for open-ended business processes
AI systems making consequential decisions without human review
Multi-model orchestration for complex reasoning tasks

Evaluate carefully before building:

Replacing human roles wholesale (usually premature and often counterproductive)
Training proprietary models (expensive, requires data infrastructure most companies don't have)
Real-time AI in latency-sensitive critical paths

The companies that will be best positioned in three years are not those who adopted AI fastest. They're the ones who adopted AI thoughtfully — building on genuine capabilities, maintaining flexibility as the landscape shifts, and solving real problems rather than demonstrating AI adoption for its own sake.

For a deeper look at how these trends are playing out across different business functions, our team at Lycore has written about the practical implications for software businesses — including what the timeline for genuine agentic automation actually looks like when you look past the marketing.

The Honest Summary

AI is changing business software meaningfully and durably. The changes are real but more incremental than the hype suggests, more dependent on data quality than vendors admit, and more constrained by organizational factors than technologists acknowledge.

The developers and engineers who will navigate this well are those who stay close to evidence — who look at what is working in production rather than what's impressive in demos, who measure adoption rather than capability, and who maintain enough technical foundation to switch approaches as the landscape evolves.

The wave is real. Riding it well requires keeping your feet on the ground.

What AI bets are you making in your current projects? I'm particularly interested in hearing from people who've tried things that didn't work — those stories are usually more instructive than the success cases.

Your Tech Stack Has an AI Problem: How to Audit and Fix It in 2026

Lycore Development — Tue, 19 May 2026 04:00:00 +0000

The Stack That Made Sense in 2022 Might Be Working Against You Now

Two years ago, the advice was consistent: pick boring technology. Rails, Django, Postgres, maybe some Redis. Proven tools, well-understood failure modes, strong hiring pools.

That advice isn't wrong. But it's incomplete in 2026, because the definition of "boring" is changing fast. The tools that were exotic in 2022 — vector databases, LLM APIs, streaming inference, semantic search — are now table stakes. And teams whose stacks weren't designed to integrate them are spending engineering cycles on plumbing rather than product.

This isn't a post about rewriting everything. It's about doing a clear-eyed audit of where your current stack creates friction for AI integration, and making targeted changes rather than wholesale replacements.

The Audit Framework: Four Layers to Examine

A tech stack audit for AI readiness covers four layers:

Data layer — Can your data be easily fed to AI systems?
Compute layer — Can you run or call inference affordably at scale?
Integration layer — Can your services consume and produce AI outputs cleanly?
Observability layer — Can you monitor AI system behaviour in production?

Let's go through each.

Layer 1: The Data Layer

AI systems are only as good as the data they operate on. The most common data layer problems we find in audits:

Unstructured data sitting in blobs with no retrieval story

You have years of customer emails, support tickets, sales calls, and internal documents in S3 or Google Drive. You know there's value in there. You have no way to query it semantically.

The fix: a vector store pipeline. Chunk the documents, embed them, store the vectors. This is now a commodity operation — pgvector on Postgres handles many use cases without a dedicated vector database.

import anthropic
import psycopg2
import json
from typing import Optional

client = anthropic.Anthropic()

def embed_text(text: str) -> list[float]:
    """Generate embeddings using a lightweight approach via Claude."""
    # In production: use a dedicated embedding model like text-embedding-3-small
    # or voyage-3 for cost efficiency. Claude isn't primarily an embedding model.
    # This is a placeholder showing the integration pattern.
    response = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=10,
        messages=[{"role": "user", "content": f"Embed: {text[:100]}"}]
    )
    # Real implementation: call your embedding API here
    return []  

def store_document_chunks(
    conn: psycopg2.extensions.connection,
    document_id: str,
    chunks: list[str],
    metadata: dict
) -> int:
    """Store document chunks with embeddings in pgvector."""
    stored = 0
    with conn.cursor() as cur:
        for i, chunk in enumerate(chunks):
            embedding = embed_text(chunk)

            cur.execute(
                """INSERT INTO document_chunks 
                   (document_id, chunk_index, content, embedding, metadata)
                   VALUES (%s, %s, %s, %s::vector, %s)
                   ON CONFLICT (document_id, chunk_index) DO UPDATE
                   SET content = EXCLUDED.content,
                       embedding = EXCLUDED.embedding""",
                (document_id, i, chunk, embedding, json.dumps(metadata))
            )
            stored += 1

    conn.commit()
    return stored

def semantic_search(
    conn: psycopg2.extensions.connection,
    query: str,
    limit: int = 5,
    metadata_filter: Optional[dict] = None
) -> list[dict]:
    """Search document chunks by semantic similarity."""
    query_embedding = embed_text(query)

    filter_clause = ""
    filter_params = []
    if metadata_filter:
        conditions = [f"metadata->>{repr(k)} = %s" for k in metadata_filter]
        filter_clause = "WHERE " + " AND ".join(conditions)
        filter_params = list(metadata_filter.values())

    with conn.cursor() as cur:
        cur.execute(
            f"""SELECT document_id, chunk_index, content, metadata,
                       1 - (embedding <=> %s::vector) AS similarity
                FROM document_chunks
                {filter_clause}
                ORDER BY embedding <=> %s::vector
                LIMIT %s""",
            [query_embedding] + filter_params + [query_embedding, limit]
        )

        return [
            {
                "document_id": row[0],
                "chunk_index": row[1],
                "content": row[2],
                "metadata": row[3],
                "similarity": float(row[4])
            }
            for row in cur.fetchall()
        ]

Schema design that doesn't support AI-generated fields

Many existing schemas were designed with the assumption that every field comes from a human or a deterministic system. AI-generated fields have different characteristics: they can be regenerated, they have confidence scores, they need provenance tracking.

A pattern we use:

-- Instead of adding AI fields directly to the parent table:
CREATE TABLE customer_ai_attributes (
    customer_id UUID REFERENCES customers(id),
    attribute_key VARCHAR(100) NOT NULL,
    attribute_value TEXT,
    confidence FLOAT,
    model_version VARCHAR(50),
    generated_at TIMESTAMPTZ DEFAULT NOW(),
    expires_at TIMESTAMPTZ,  -- AI outputs can go stale
    PRIMARY KEY (customer_id, attribute_key)
);

-- This allows you to:
-- 1. Update AI attributes independently from the customer record
-- 2. Track which model version produced each attribute
-- 3. Expire stale AI outputs and regenerate them
-- 4. Roll back to previous AI-generated values if a model update regresses

Missing event streams

AI systems often need real-time data — not batch exports from your OLAP warehouse. If your architecture doesn't have an event stream (Kafka, Kinesis, Azure Service Bus), adding AI features that react to real-time events is painful.

This doesn't mean you need Kafka on day one. For many applications, Postgres + a polling worker is sufficient. But if you're seeing requirements like "update the AI recommendation when the user's behaviour changes," you need to think about your event story.

Layer 2: The Compute Layer

The question here is simple: where does the inference run, and what does it cost at your projected scale?

The build vs. buy matrix for AI compute

Use Case	Recommended Approach	Why
Chat/generation features	API (Anthropic, OpenAI)	Cost-efficient at most scales; managed availability
High-volume classification	Fine-tuned small model, self-hosted	Frontier APIs get expensive at millions of calls/day
Embedding generation	Dedicated embedding API or self-hosted	voyage-3, text-embedding-3-small are cost-optimised for this
Image/audio processing	Specialist APIs	Don't build what Whisper or vision APIs already do well
Sensitive data processing	Self-hosted open-source model	Data sovereignty requirements may prohibit API calls

The compute audit question: are you using frontier API calls for tasks where a smaller, cheaper model would be sufficient? Over-indexing on GPT-4 class models for classification, routing, and summarisation is one of the most common AI cost problems.

Caching strategy

Many AI applications call the same prompts with the same inputs repeatedly. Without caching, you're paying for the same computation over and over.

Anthropic's prompt caching (available via the API) can reduce costs by 90%+ on repeated long-context calls. For application-level caching:

import hashlib
import json
import redis
from anthropic import Anthropic

class CachedAnthropicClient:
    """
    Wrapper around Anthropic client with Redis caching.
    Appropriate for deterministic or near-deterministic use cases.
    """

    def __init__(self, cache_ttl_seconds: int = 3600):
        self.client = Anthropic()
        self.cache = redis.Redis()
        self.ttl = cache_ttl_seconds

    def cached_complete(self, model: str, messages: list, system: str = "", max_tokens: int = 1024, temperature: float = 0) -> str:
        """
        Complete with caching. Only cache when temperature=0 (deterministic).
        """
        if temperature > 0:
            # Don't cache non-deterministic outputs
            return self._complete(model, messages, system, max_tokens, temperature)

        cache_key = self._make_cache_key(model, messages, system, max_tokens)

        cached = self.cache.get(cache_key)
        if cached:
            return json.loads(cached)

        result = self._complete(model, messages, system, max_tokens, temperature)
        self.cache.setex(cache_key, self.ttl, json.dumps(result))
        return result

    def _complete(self, model, messages, system, max_tokens, temperature) -> str:
        kwargs = {"model": model, "max_tokens": max_tokens, "messages": messages}
        if system:
            kwargs["system"] = system
        response = self.client.messages.create(**kwargs)
        return response.content[0].text

    def _make_cache_key(self, model: str, messages: list, system: str, max_tokens: int) -> str:
        payload = json.dumps({"model": model, "messages": messages, "system": system, "max_tokens": max_tokens}, sort_keys=True)
        return f"llm_cache:{hashlib.sha256(payload.encode()).hexdigest()}"

Layer 3: The Integration Layer

This is where most stacks have the most friction. The question is: how easily can your existing services consume AI outputs and produce AI inputs?

The API contract problem

AI outputs are probabilistic and variable. Your existing services probably expect deterministic, well-typed inputs. The integration layer needs to handle the translation.

Patterns that work:

Strict output schemas: Use structured outputs (JSON mode, tool use for output parsing) to ensure AI outputs conform to your internal data contracts. Never pass raw LLM text directly to downstream services.

Async processing with status tracking: AI calls are slower and less predictable than database queries. Don't make synchronous AI calls in request paths where latency matters. Use job queues, return a job ID immediately, and let clients poll or subscribe to updates.

Graceful degradation: Every AI integration should have a defined fallback. If the AI call fails or times out, what does the system do? Return a default, surface a rule-based fallback, or fail gracefully with a clear user-facing message.

The LLM framework question

In 2024, the advice was "use LangChain." In 2026, the advice is more nuanced.

LangChain and LlamaIndex are powerful frameworks with large ecosystems. They're also complex, and that complexity has costs: debugging is harder, upgrade paths are painful, and the abstraction layer can obscure what's actually happening in your LLM calls.

For teams doing a tech stack audit, we recommend a fresh evaluation of your LLM framework choices based on actual requirements. The questions to ask:

Are you using 20% of the framework's features? (Common — most teams are)
Is the framework version compatible with the LLM APIs you need? (Breaking changes are frequent)
Could you replace the framework usage with direct API calls and a small utility library?

For many use cases, direct API calls with a thin abstraction layer are more maintainable than a full framework dependency. For complex RAG pipelines and multi-agent systems, framework tooling earns its place.

Layer 4: Observability

You cannot operate AI systems in production without visibility into what they're doing, how much they cost, and when they break.

What good AI observability looks like

Cost tracking per feature: You need to know which feature is driving your AI API spend. "Claude API cost" as a single line item is useless. You need "recommendation engine: $X/day, search: $Y/day, support chatbot: $Z/day."

import time
from anthropic import Anthropic
from dataclasses import dataclass

@dataclass
class LLMCallMetrics:
    feature: str
    model: str
    input_tokens: int
    output_tokens: int
    latency_ms: int
    cached: bool = False

class InstrumentedAnthropicClient:
    """Anthropic client with cost and latency tracking per feature."""

    COST_PER_MILLION = {
        "claude-sonnet-4-20250514": {"input": 3.0, "output": 15.0},
        "claude-haiku-4-5-20251001": {"input": 0.25, "output": 1.25},
    }

    def __init__(self, metrics_emitter):
        self.client = Anthropic()
        self.metrics = metrics_emitter  # Your metrics system (Datadog, Prometheus, etc.)

    def complete(self, feature: str, model: str, messages: list, **kwargs) -> str:
        start = time.time()

        response = self.client.messages.create(
            model=model, messages=messages, **kwargs
        )

        latency_ms = int((time.time() - start) * 1000)

        m = LLMCallMetrics(
            feature=feature,
            model=model,
            input_tokens=response.usage.input_tokens,
            output_tokens=response.usage.output_tokens,
            latency_ms=latency_ms
        )

        # Emit metrics tagged by feature
        self.metrics.histogram("llm.latency_ms", latency_ms, tags=[f"feature:{feature}", f"model:{model}"])
        self.metrics.increment("llm.input_tokens", m.input_tokens, tags=[f"feature:{feature}"])
        self.metrics.increment("llm.output_tokens", m.output_tokens, tags=[f"feature:{feature}"])

        cost = self._calculate_cost(model, m.input_tokens, m.output_tokens)
        self.metrics.gauge("llm.cost_usd", cost, tags=[f"feature:{feature}"])

        return response.content[0].text

    def _calculate_cost(self, model: str, input_tokens: int, output_tokens: int) -> float:
        rates = self.COST_PER_MILLION.get(model, {"input": 3.0, "output": 15.0})
        return (input_tokens * rates["input"] + output_tokens * rates["output"]) / 1_000_000

The Audit Output: A Prioritised Action List

After running this audit with clients, we typically produce a prioritised action list across four categories:

Quick wins (1-2 weeks): Usually caching, cost attribution tagging, and structured output enforcement. These reduce cost and improve reliability without architectural changes.

Medium-term improvements (1-3 months): Typically the data layer — setting up vector stores, building event streams, adding AI-attribute tables to the schema.

Strategic changes (3-6 months): Framework evaluations, compute architecture decisions, self-hosting assessments for high-volume use cases.

Future-proofing (ongoing): Staying current with model API changes, running regular cost/performance benchmarks, and maintaining the ability to swap model providers without rewriting application code.

If you're at a point where you know AI needs to be more central to your product but your current stack is creating friction, a focused tech stack audit is usually the right first step. It tells you exactly what to change, in what order, and what it will cost — rather than the more expensive path of discovering the problems one at a time as you build.

Have you done a tech stack audit for AI readiness? What did you find? I'm curious whether the patterns we see are consistent across different team sizes and industries.