grimoire/PROOF_OF_CONCEPT_LLM_CHAT.md

Proof of Concept: LLM Chat Using Generic Components

This document demonstrates how the generic chat components can be used for a completely different chat paradigm: AI/LLM conversation interfaces.

Overview

We've successfully created an LLM chat interface using the same generic components originally extracted from the Nostr chat implementation. This proves the abstraction is truly protocol-agnostic and reusable.

Try It

# Open the LLM chat interface
llm

Key Differences: Nostr Chat vs LLM Chat

Aspect	Nostr Chat	LLM Chat
Participants	Multi-user (groups, DMs)	1-on-1 (user ↔ assistant)
Message Flow	Real-time, event-based	Request-response pattern
Streaming	N/A	Token-by-token streaming
Infrastructure	Decentralized relays	Centralized API
Protocol Adapters	NIP-29, NIP-53, etc.	OpenAI, Anthropic, local models
Message Types	User messages, system events, zaps	User, assistant, system prompts
Metadata	Nostr events, signatures, reactions	Tokens, costs, model info
Special Features	Relay selection, zaps, reactions	Model selection, temperature, streaming

Architecture

Generic Components (Shared)

All in src/components/chat/shared/:

ChatWindow        - Main layout (header + list + composer)
MessageList       - Virtualized scrolling with day markers
MessageComposer   - Input with autocomplete support
ChatHeader        - Flexible header layout
DayMarker         - Date separators
date-utils        - Date formatting & marker insertion
types.ts          - Generic TypeScript interfaces

Protocol-Specific Implementations

Nostr Chat (src/components/ChatViewer.tsx):

• Integrates with Nostr event store
• Uses protocol adapters (NIP-29, NIP-53)
• Renders Nostr-specific UI (UserName, RichText, zaps)
• Manages relay connections
• Handles Nostr-specific features (reactions, zaps, moderation)

LLM Chat (src/components/LLMChatViewer.tsx):

• Manages conversation state locally
• Uses provider adapters (Mock, OpenAI, Anthropic, etc.)
• Renders LLM-specific UI (streaming indicator, token count, cost)
• Handles streaming responses
• Shows model selection and settings

Code Reuse Metrics

• 100% of layout components reused
• 100% of virtualization logic reused
• 100% of day marker logic reused
• 100% of message composer UI reused
• 0% of protocol-specific logic shared (as intended)

Implementation Details

LLM Message Type

interface LLMMessage {
  id: string;
  role: "user" | "assistant" | "system";
  content: string;
  timestamp: number;
  streaming?: boolean;      // Token-by-token streaming
  tokens?: number;          // Token count
  cost?: number;            // USD cost
  model?: string;           // Model used
  error?: string;           // Error message
}

Compatible with generic DisplayMessage interface via structural typing.

Provider Adapter Pattern

interface LLMProviderAdapter {
  provider: LLMProvider;
  sendMessage(
    messages: LLMMessage[],
    settings: LLMConversationSettings,
    onChunk?: (chunk: LLMStreamChunk) => void,
  ): Promise<LLMMessage>;
  validateAuth(apiKey: string): Promise<boolean>;
  countTokens?(text: string, model: string): Promise<number>;
}

Similar to Nostr's ChatProtocolAdapter pattern.

Mock Provider

For demonstration, we created a mock provider that:

• Simulates streaming responses word-by-word
• Provides canned responses with code examples
• Estimates token counts
• Has zero cost (it's fake!)
• No API key required

Streaming Implementation

// Add streaming message placeholder
const streamingMessage: LLMMessage = {
  id: `assistant-${Date.now()}`,
  role: "assistant",
  content: "",
  timestamp: Date.now() / 1000,
  streaming: true,
};

// Update message as chunks arrive
await provider.sendMessage(messages, settings, (chunk) => {
  setConversation((prev) => {
    const messages = [...prev.messages];
    const lastMessage = messages[messages.length - 1];
    messages[messages.length - 1] = {
      ...lastMessage,
      content: lastMessage.content + chunk.content,
      streaming: !chunk.done,
    };
    return { ...prev, messages };
  });
});

Generic Component Features Used

From ChatWindow

• ✅ Loading/error states
• ✅ Header with custom content
• ✅ Header prefix/suffix areas
• ✅ Message virtualization
• ✅ Empty state
• ✅ Composer integration

From MessageList

• ✅ Infinite scroll
• ✅ Day markers
• ✅ Custom message rendering

From MessageComposer

• ✅ Text input
• ✅ Send button
• ✅ Disabled states
• ⚠️ Autocomplete (not used in this demo)
• ⚠️ Attachments (not used in this demo)

From date-utils

• ✅ insertDayMarkers() - Works perfectly with LLM messages
• ✅ formatDayMarker() - "Today", "Yesterday", "Jan 15"
• ✅ isDifferentDay() - Day comparison

Future Enhancements for LLM Chat

Planned Features

• Real provider integrations (OpenAI, Anthropic, local models)
• Model switching mid-conversation
• System prompt editor
• Temperature/settings controls
• Code syntax highlighting (using react-syntax-highlighter)
• Message editing and regeneration
• Conversation export (JSON, markdown)
• Token usage tracking and cost estimates
• Conversation persistence (localStorage, Dexie)
• Multiple conversation tabs
• Search within conversation

Possible Provider Implementations

• OpenAI Provider: GPT-4, GPT-3.5, with function calling support
• Anthropic Provider: Claude 3 Opus, Sonnet, Haiku with streaming
• Local Provider: Ollama, LM Studio, llama.cpp
• Custom Provider: Any API implementing the adapter interface

Benefits of This Abstraction

1. Rapid Prototyping: Built LLM chat in < 300 lines of code
2. Type Safety: Full TypeScript support across protocols
3. Performance: Virtualization works for any message type
4. Consistency: Same UX patterns across different chat types
5. Maintainability: Bug fixes in shared components benefit all implementations
6. Flexibility: Easy to add new chat protocols (Matrix, XMPP, IRC, Discord, etc.)

Comparison with Other Implementations

Traditional Approach (No Abstraction)

// Would need to reimplement:
- Virtualized scrolling
- Day marker logic
- Message composer
- Loading states
- Header layout
Total: ~800 lines duplicated

With Generic Components

// Only implement:
- Message rendering (50 lines)
- Protocol adapter (100 lines)
- State management (150 lines)
Total: ~300 lines unique code

Code Reduction: 62%

Conclusion

The generic chat components successfully abstract the UI layer from the protocol layer, enabling:

• Different message sources (Nostr relays, LLM APIs, WebSockets, etc.)
• Different message types (events, responses, notifications, etc.)
• Different interaction patterns (multi-user, 1-on-1, streaming, etc.)
• Different metadata (signatures, tokens, timestamps, etc.)

This proves the refactoring achieved its goal of creating truly reusable chat components that work across completely different chat paradigms.

Files Added

src/lib/llm/
├── types.ts                           # LLM-specific types
└── providers/
    └── mock-provider.ts               # Mock provider for demo

src/components/
└── LLMChatViewer.tsx                  # LLM chat UI using generic components

Try Building Your Own

Want to add a new chat protocol? Here's what you need:

1. Define message type that extends { id: string; timestamp: number }
2. Create provider adapter with sendMessage() implementation
3. Build message renderer as a React component
4. Wire into ChatWindow with render props
5. Add command to man.ts and WindowRenderer.tsx

That's it! The generic components handle everything else.