Optimizing LLM Integrations in React

Where the money goes

An LLM call takes anywhere from one to several seconds and bills you per token in both directions. Neither problem can be fixed in the UI layer, but both can be made much less annoying there, and that turns out to be most of the job.

Render the user's message immediately

Never make the user wait for the model to acknowledge their own input. Append their message to local state the moment they hit enter, then stream the reply into a placeholder. If the model is about to do something predictable, render the expected result optimistically and reconcile when the response lands. The API is slow. Your state updates are not.

Debounce anything typed

Wire a model to autocomplete without a debounce and you will learn exactly what your per-request pricing looks like at one call per keystroke. A second of silence is a decent threshold, since people pause when they actually want a suggestion.

import { useDebounce } from 'use-debounce';

const [text, setText] = useState('');
const [query] = useDebounce(text, 1000);

useEffect(() => {
  if (!query) return;
  const controller = new AbortController();
  fetchSuggestion(query, controller.signal);
  return () => controller.abort();
}, [query]);

The AbortController is not optional. A stale suggestion arriving after a newer one is worse than no suggestion at all.

Suspense, where it fits

Wrapping model-dependent components in <Suspense> keeps the rest of the page interactive while inference runs, and the fallback gives you a free loading state. One caveat: Suspense is for the initial wait, not for streaming. Token-by-token chat output is incremental state, so handle it with a streaming hook rather than trying to bend Suspense into a job it was not built for.

Count tokens before you send

A rough client-side estimate (characters divided by four gets you surprisingly far) lets you warn users their prompt is over budget before the request fails server-side. If you need the real number, a proper tokenizer runs fine in the browser.

None of this is new. It is the same optimistic-update, debounce, and cancellation discipline React developers have practiced for years, pointed at an API that is slower and more expensive than the ones it was invented for.