Formflow Ai Assistant

formflow-ai-assistant

A React chatbot widget for form-driven applications. It converts natural language into structured actions that can fill react-hook-form fields, navigate tabs, and add or update table rows.

What This Package Owns

• Chat UI: AIAssistantWidget, ChatPanel, message rendering.
• AI orchestration: prompt building, Gemini/OpenAI request handling, response parsing.
• Action execution: field updates, tab navigation, table handlers.
• UI-library adapters: DOM behavior for controls that cannot be updated safely with direct form state writes, such as async dropdowns.

App/domain knowledge stays outside the package in an AIAssistantConfig.

Installation

npm install formflow-ai-assistant

Quick Start

1. Create App Config

import type { AIAssistantConfig } from 'formflow-ai-assistant';

export const aiConfig: AIAssistantConfig = {
  appType: 'shipment',
  appDescription: 'Freight shipment management for ocean and air cargo',
  fields: {
    transportMode: {
      formPath: 'transportMode',
      label: 'Transport Mode',
      type: 'dropdown',
      options: [
        { value: 'SEA', aliases: ['sea', 'ocean'] },
        { value: 'AIR', aliases: ['air', 'flight'] },
      ],
    },
    client: {
      formPath: 'client',
      label: 'Client',
      type: 'async-dropdown',
      description: 'Customer/client for this shipment',
    },
  },
  tabs: {
    main: [
      { id: 'shipmentDetails', label: 'Shipment Details', aliases: ['details'] },
      { id: 'cargo', label: 'Cargo', aliases: ['cargo', 'containers'] },
    ],
  },
};

2. Mount Provider and Widget

import { FormProvider, useForm } from 'react-hook-form';
import {
  AIAssistantWidget,
  AIAssistantProvider,
  defaultUIAdapter,
} from 'formflow-ai-assistant';
import { aiConfig } from './ai-assistant/config';

function ShipmentForm() {
  const formMethods = useForm();
  const [currentTab, setCurrentTab] = useState('shipmentDetails');

  return (
    <AIAssistantProvider
      config={aiConfig}
      formMethods={formMethods}
      tabNavigator={{
        currentTab,
        setTab: setCurrentTab,
      }}
      uiAdapter={defaultUIAdapter}
      geminiApiKey={process.env.REACT_APP_GEMINI_API_KEY}
      provider="gemini"
      model="gemini-flash-latest"
    >
      <FormProvider {...formMethods}>{/* app */}</FormProvider>
      <AIAssistantWidget position="bottom-right" />
    </AIAssistantProvider>
  );
}

UI-Library Adapters

The assistant is app-config driven, but some UI controls need real DOM interaction. Async dropdowns are the main case: direct setValue can miss option metadata or side effects, so the assistant opens the dropdown, searches, waits for API results, and selects a result.

UI-library-specific behavior lives under:

src/ui-libs/<ui-library>/

Current built-in adapter:

import { defaultUIAdapter } from 'formflow-ai-assistant';

Adapter Interface

interface UIAdapterConfig {
  asyncDropdown?: {
    getTrigger: (field: FieldConfig, formPath: string) => HTMLElement | null;
    getPopover: () => HTMLElement | null;
    getSearchInput: (popover: HTMLElement) => HTMLInputElement | null;
    getOptions: (popover: HTMLElement) => HTMLElement[];
    openWaitMs?: number;
    searchWaitMs?: number;
    afterSelectWaitMs?: number;
  };
}

The default adapter uses common accessible DOM selectors:

• trigger: document.getElementById(formPath)
• popover: latest [role="listbox"]
• search input: first input inside popover
• options: [role="option"]
• waits: 2000ms, 2000ms, 100ms

Async dropdown action behavior:

1. Open dropdown.
2. Wait for initial async options.
3. Type the requested value in the search input.
4. Wait for searched API results.
5. Select the first returned option.

To add another UI library, create a folder such as src/ui-libs/mui/, implement the same adapter interface, export it from src/ui-libs/index.ts, then pass it as uiAdapter from the host app.

Provider Props

interface AIAssistantProviderProps {
  config: AIAssistantConfig;
  formMethods: UseFormReturn<Record<string, unknown>>;
  tabNavigator: TabNavigator;
  tableHandlers?: Record<string, TableHandler>;
  uiAdapter?: UIAdapterConfig;
  openAIApiKey?: string;
  geminiApiKey?: string;
  claudeApiKey?: string; // deprecated alias
  provider?: 'openai' | 'gemini';
  model?: string;
  additionalSystemPrompt?: string;
  onActionExecuted?: (action: AIAction, success: boolean) => void;
  onError?: (error: Error) => void;
  children: React.ReactNode;
}

Tables

Use react-hook-form field-array handlers for table actions:

const { append, update } = useFieldArray({
  control: formMethods.control,
  name: 'containersList',
});

<AIAssistantProvider
  tableHandlers={{
    containers: { append, update },
  }}
  // ...
/>

Using Another Model or Training a Future Model

There are two different paths: using another hosted model, or training/fine-tuning a model to produce this app's action JSON more reliably.

Use Another Hosted Model

1. Add provider support in the AI request hook (src/hooks/useOpenAIAPI.ts, or split it into provider-specific clients later).
2. Keep the output contract unchanged: the model must return JSON with:

   {
     "message": "Human-readable response",
     "actions": []
   }
   

3. Add a provider name to the provider union type.
4. Add provider-specific API key prop if needed, for example anthropicApiKey or localModelUrl.
5. Map the provider response text into parseAIResponse.
6. Build and test with the same app config and examples.

Fine-Tune or Train a Model for This App

1. Keep the action schema stable. The important training target is AIResponse and AIAction, not the UI.
2. Collect examples from real usage:
   • user request
   • relevant current form context
   • expected message
   • expected actions
3. Store examples in the same shape as ConversationExample where possible.
4. Include negative examples where the model should return CLARIFY instead of guessing.
5. Train/fine-tune the model to output only valid JSON matching the action schema.
6. Validate every model response with parseAIResponse before executing actions.
7. Keep UI library logic out of training data. The model should decide what action to perform; the adapter decides how the UI executes it.
8. Add regression prompts for core flows: static dropdowns, async dropdowns, tab navigation, table row creation, and ambiguous requests.

Recommended starting point: do not fine-tune immediately. First improve examples in the app config and add provider support. Fine-tune only after you have enough failed prompts or repeated workflows to justify it.

Security

Browser env keys such as REACT_APP_GEMINI_API_KEY are exposed to users. This is acceptable for local testing only. For production, route model calls through a backend proxy and keep provider keys server-side.

Troubleshooting

• If static fields do not update, verify formPath matches react-hook-form state.
• If async dropdowns do not work, verify the correct uiAdapter is passed.
• If async dropdown opens but does not select, inspect the UI library's trigger, popover, search input, and option selectors.
• If the model returns bad JSON, add examples or tighten additionalSystemPrompt.
• If actions are valid but fail, check onActionExecuted and browser console warnings.

Validation

import { validateConfig, formatValidationErrors } from 'formflow-ai-assistant';

const errors = validateConfig(aiConfig);
if (errors.length > 0) {
  console.error(formatValidationErrors(errors));
}

License

MIT