Product Architecture
Complete system architecture and workflow overview
Product Architecture
Table of Contents
- High-Level Architecture
- Six-Phase Workflow
- Two Question Systems
- Data Flow
- User Stories System
- Phase Transitions
- Key Services
- Critical Files
High-Level Architecture
┌─────────────────────────────────────────────────────────────────────────────┐
│ FRONTEND (Next.js 15) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Conversation│ │ Project │ │ Task │ │ Form Wizard System │ │
│ │ Pages │ │ Dashboard │ │ Management │ │ (23+ field types) │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────────┬──────────┘ │
└─────────┼────────────────┼────────────────┼────────────────────┼────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ API LAYER (Hono) │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ /api/flow │ │/api/projects│ │ /api/tasks │ │ /api/agents │ │
│ │ /api/conv │ │/api/features│ │/api/actions │ │ /api/specification │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────────┬──────────┘ │
└─────────┼────────────────┼────────────────┼────────────────────┼────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ SERVICE LAYER (220+ files) │
│ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────────────┐│
│ │ Flow Services │ │ Feature Services │ │ Planning Services ││
│ │ - Question Flow │ │ - Selection │ │ - Plan Generation ││
│ │ - Variant Enhance │ │ - Validation │ │ - Readiness Assessment ││
│ │ - Ontology Track │ │ - Scoring │ │ - Phase Transitions ││
│ │ - Answer Prefill │ │ - Curation │ │ - Blueprint Generation ││
│ │ - Persona Match │ └───────────────────┘ └───────────────────────────┘│
│ └───────────────────┘ │
│ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────────────┐│
│ │ Task Services │ │ Design Services │ │ Background Services ││
│ │ - Task Detection │ │ - Design Decisions│ │ - Worker (10s poll) ││
│ │ - Task Generation │ │ - System Compile │ │ - Post-Answer Processing ││
│ │ - Output Binding │ │ - Preference │ │ - Feature Curation ││
│ └───────────────────┘ └───────────────────┘ └───────────────────────────┘│
└─────────────────────────────────────────────────────────────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ ENTERPRISE SYSTEMS LAYER (60+ files) │
│ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────────────┐│
│ │Permissions System │ │Document Templates │ │ BPMN Workflows ││
│ │ - RBAC + ABAC │ │ - Variable Extract│ │ - Parsing & Validation ││
│ │ - Policy Engine │ │ - Mapping & Render│ │ - Normalization ││
│ └───────────────────┘ └───────────────────┘ └───────────────────────────┘│
│ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────────────┐│
│ │ Integrations │ │ Schema Registry │ │ Gaps Analysis ││
│ │ - Curated + Custom│ │ - Domain Objects │ │ - Detector Engine ││
│ │ - Contract Import │ │ - Versioning │ │ - Task Synthesis ││
│ └───────────────────┘ └───────────────────┘ └───────────────────────────┘│
│ ┌───────────────────┐ │
│ │ Export Compiler │ │
│ │ - Bundle Gen │ │
│ │ - Readiness Gate │ │
│ └───────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ CORE LAYER (Orchestration) │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ PHASE ORCHESTRATOR │ │
│ │ (src/services/flow/phase-orchestrator.ts - THE MOST IMPORTANT FILE) │ │
│ │ │ │
│ │ - Phase lifecycle management │ │
│ │ - Transition validation │ │
│ │ - Readiness calculation │ │
│ │ - Entry action execution │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ Agent System │ │ Graph System │ │ Ontology System │ │
│ │ (17 agents) │ │ (Q&A flow) │ │ (domain knowledge) │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ MASTRA WORKFLOW LAYER │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ Phase Workflows │ │ Spec Workflows │ │ Research Workflows │ │
│ │ - Transitions │ │ - compile-spec │ │ - market-research │ │
│ │ - Entry Actions │ │ - multi-agent │ │ - design-intelligence │ │
│ │ - Validation │ │ - variant-build │ │ - technical-feasibility │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ DATA LAYER (PostgreSQL) │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ Project Tables │ │ Question Tables │ │ Agent/Execution Tables │ │
│ │ - projects │ │ - graph_nodes │ │ - agent_memory │ │
│ │ - tenants │ │ - plan_questions│ │ - agent_executor_state │ │
│ │ - users │ │ - answers │ │ - checkpoint_snapshots │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
│ Row-Level Security (RLS) Enforced │
└─────────────────────────────────────────────────────────────────────────────┘Six-Phase Workflow
The Praetor system operates on a 6-phase model that progressively refines user intent into production-ready specifications:
┌─────────────────────────────────────────────────────────────────────────────┐
│ │
│ PHASE 1 PHASE 2 PHASE 3 PHASE 4 │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │DISCOVERY│ ──▶ │ FEATURE │ ──▶ │ PLAN │ ──▶ │PLANNING │ │
│ │ │ │SELECTION│ │GENERATION│ │ │ │
│ │ ~15 Qs │ │ │ │ 50-150 │ │Blueprint│ │
│ │template │ │ select │ │questions│ │ & Plan │ │
│ │ driven │ │features │ │AI-gen'd │ │ │ │
│ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │
│ │ │ │
│ │ ▼ │
│ │ ┌─────────┐ │
│ │ PHASE 6 PHASE 5 │ VISUAL │ │
│ │ ┌─────────┐ ┌─────────┐ │DISCOVERY│ │
│ └────────▶ │ SPEC │ ◀──── │ VISUAL │ ◀──── │ │ │
│ (terminal) │(TERMINAL)│ │DISCOVERY│ │ design │ │
│ │ │ │ │ │questions│ │
│ │ review │ │ 2 tracks│ │ + AI │ │
│ │ export │ │parallel │ │research │ │
│ └─────────┘ └─────────┘ └─────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘Phase 1: DISCOVERY
Purpose: Understand WHAT the user wants to build
Two Discovery Systems (Hybrid Approach):
1. Requirements Discovery (New - Migration 094)
Conversational requirements gathering with structured extraction:
- Entry:
/api/requirements-discovery/startfor new projects - Format: Natural conversation, not fixed questions
- Storage:
requirements_discovery_contextstable with structured JSONB fields - Process: AI agent conducts guided conversation, extracts structured data in real-time
- Extraction: 7 core fields with confidence scoring:
purpose_and_problem: Core problem, trigger event, current solutionusers_and_personas: Primary users, jobs to be done, pain pointssuccess_criteria: Measures of success, dealbreakersconstraints: Timeline, budget, technical constraints, existing systemsnon_functional_screening: Sensitive data, expected scaledomain_specific: Optional domain questions (e.g., e-commerce, SaaS)
- Audit Trail: Every extraction logged in
requirements_discovery_extractionswith:- Field path (e.g., "purpose_and_problem.coreProblem")
- Confidence score (0.0-1.0)
- Source message snippet
- Extraction type (fact/decision/inference)
Key Benefits:
- More natural for users (conversation vs form)
- Structured data enables intelligent feature matching
- Confidence scores highlight areas needing clarification
- Full audit trail shows what was captured
2. Graph-Based Discovery (Legacy - Still Supported)
Template-driven questions with graph structure:
- Entry:
/api/flow/startcalled on project - Questions: ~15 template-driven questions stored in
graph_nodestable - Process: Template instantiation creates questions based on project type
- Readiness:
answered_questions / total_questionsratio - Storage: Unstructured key-value answers in
graph_nodes.content
Hybrid Bridge:
discovery-context-bridge.tsservice maps both formats to unifiedDiscoveryContextinterface- Requirements Discovery takes precedence when both exist
- Falls back to graph-based if no requirements discovery context exists
Exit Trigger: Discovery complete (requirements extracted OR graph questions answered)
Exit Actions:
- Generate
feature_seedsfrom structured context (for prioritization) - Create
feature_curation_sessionlinking discovery → feature selection - Activate relevant question pools based on detected signals (e.g., "Stripe" → payment pool)
- Generate feature candidates using enhanced feature matcher
Technical Details:
- Pool Activation:
post-message-processor.tsactivates pools dynamically during discovery- Monitors each user message for technology mentions
- Calls
DatabasePoolActivationRules.analyzeForActivation() - Injects pool questions when high-confidence match (>0.8)
- Feature Matching: Uses structured signals instead of keyword matching:
- Goals, pain points, jobs to be done
- Constraints, dealbreakers, sensitive data
- Existing systems, expected scale
- Extraction Service:
extraction-service.tsanalyzes each message, updates context incrementally - Ontology Tracker: Monitors entity coverage in real-time (graph-based discovery)
Phase 2: FEATURE_SELECTION
Purpose: User selects which features to include in their project
- Entry: Feature candidates already generated from discovery
- Storage:
feature_selectionstable withfeature_candidate_sets - Process: User reviews AI-suggested features, adds/removes as needed
- Exit Trigger: Feature selection confirmed
- Exit Action: Generate feature-driven question plan
Technical Details:
- Feature candidates scored by relevance and complexity
- Selection state tracked in
feature_selection_state - Feature signals (80+ keywords) used for intelligent suggestions
Phase 3: PLAN_GENERATION
Purpose: Generate detailed questions based on selected features
- Entry: Question plan generated with N questions
- Questions: 50-150+ questions stored in
plan_questionstable (NOT graph_nodes) - Process: AI generates feature-specific questions
- Prefill: Two-pass AI prefill assigns confidence scores
- Exit Trigger: All plan questions answered or prefilled above threshold
- Exit Action: Generate domain blueprint + implementation plan
Technical Details:
- Questions generated by AI based on feature requirements
- Prefill System (critical for UX):
- First pass: Generate initial answer candidates
- Second pass: Refine answers with context from other answers
- Confidence scoring: 0.0-1.0 scale
- Low-confidence items (
<0.7) become human review tasks
Phase 4: PLANNING
Purpose: Generate blueprint and implementation plan
- Entry: Blueprint and plan artifacts generated
- Artifacts:
- Domain Blueprint (architecture decisions)
- Implementation Plan (execution roadmap)
- Process: User reviews and refines AI-generated plans
- Exit Trigger: User approves artifacts
- Exit Action: Generate block requirements + design question plan
Phase 5: VISUAL_DISCOVERY
Purpose: Gather design preferences and conduct visual research through two parallel tracks
Track 1: Design Questions (User-Facing)
Design questions help users select visual components and styles for their application. Questions are organized into 7 sections:
- Theme & Overall Style - Brand identity, color schemes, overall aesthetic
- Authentication Pages - Login, signup, password reset designs
- Marketing & Landing Pages - Homepage, feature pages, CTAs
- Navigation & Layout - Header, sidebar, menu structures
- Dashboard & Data Display - Charts, tables, metrics cards
- Dialogs & Notifications - Modals, toasts, alerts, confirmations
- E-commerce Components - Product cards, checkout flows, cart UI
Data Source: Questions are generated from the Block Catalog (BlockCatalogService), NOT from question pools. Each block in the catalog represents a reusable UI component with:
- Screenshots/visual examples
- Complexity levels (basic, standard, advanced)
- Component hierarchy (parent-child relationships)
- Feature associations
UI Components:
block-card-selector- Displays screenshot options for each design surfacetheme-color-picker- Color palette selection with live previewtheme-font-picker- Typography pairing selection
Process: Each question maps a design surface (e.g., "Login Page", "Product Card") to available blocks from the catalog, sorted by complexity. Users can select multiple options per surface, with the system tracking preferences and generating a cohesive design system.
Storage: Design question plan stored in projects.design_question_plan (JSONB), with answers tracked in standard question/answer tables.
Track 2: Research (AI Background Tasks)
Visual research tasks run in parallel with user interactions, gathering competitive intelligence and design patterns.
Tier 1 Tasks (8 tasks, need 6/8 complete for progress):
- market-research - Industry trends, target audience preferences
- design-intelligence - Current design patterns, UI/UX best practices
- technical-feasibility - Component complexity, implementation effort
- competitive-analysis - Competitor visual design analysis
- visual-curation - Inspiration boards, color palettes, typography
- architecture-analysis - Design system structure, component hierarchy
- business-strategy - Brand positioning, visual differentiation
- product-management - Feature prioritization impact on UI
Tier 2 Tasks (6 tasks, need 6/6 complete, triggered after Tier 1):
- color-palette-generation - Generate cohesive color schemes
- typography-generation - Font pairing recommendations
- layout-generation - Grid systems, spacing scales
- component-generation - Suggest component library structure
- spacing-generation - Spacing tokens, rhythm systems
- style-preference-generation - Consolidated style guide
Coordination: Both tracks run in parallel. Phase readiness is weighted:
- Tier 1 Research: 40% (6 of 8 tasks complete)
- Tier 2 Research: 40% (all 6 tasks complete)
- Design Discovery Interactions: 20% (user engagement with questions)
Exit Trigger: Readiness >= 80% (configurable threshold) Exit Action: Compile visual discovery output, generate consolidated design system specification
Phase 6: SPECIFICATION (Terminal)
Purpose: Final review, validation, and export of complete specification
- Entry: All research complete, planning artifacts generated, visual discovery finished
- Status: Terminal phase (no further transitions)
- Entry Action: Trigger user story task chain (backlog_draft → spec_coverage_validator)
Readiness Check (Export Blocking)
Before export, the ReadinessChecker validates:
- Blocking Tasks: No tasks with
blocking=truein non-terminal status (DONE/SKIPPED/CANCELLED) - Blocker-Severity Gap Findings: No gap findings with
severity='blocker'andresolution_status='open' - Artifact Validation:
- Project spec exists (
project_specstable) - Design version saved (
design_versionstable) - Feature selection completed (
feature_selectionstable)
- Project spec exists (
- Phase Completion: All required reviews complete, ontology completeness verified
Each failed check creates a ReadinessBlocker that prevents export. User must resolve all blockers before export can proceed.
Export Generation
Export process supports multiple target platforms and output formats:
Generators:
DirectExportGenerator- Synchronous export for v1 formatEnhancedSpecGenerator- Asynchronous export with extended artifacts (v2 format)
Target Platforms:
claude-code- Claude Code CLI project formatcursor- Cursor IDE project formatbolt- Bolt.new formatlovable- Lovable.dev formatgeneric- Platform-agnostic format
Export Options:
format: 'v1' (legacy) or 'v2' (enhanced with ArchiMate)includeArchiMate: Include enterprise architecture layersincludeAgentsMd: Include AGENTS.md coordination fileoutputFormat: 'json' | 'markdown' | 'yaml'
Output Artifacts
Complete export bundle includes:
-
Project Specification (SpecV1 or SpecV2)
- Project metadata, goals, constraints
- Selected features with implementation details
- Technical requirements and dependencies
-
Architecture Overview (ArchiMate layers)
- Business layer (capabilities, processes, services)
- Application layer (components, interfaces, data objects)
- Technology layer (infrastructure, platforms, networks)
-
AGENTS.md - AI agent coordination file
- Project context and goals
- Feature implementation roadmap
- Design system specifications
- Technical constraints and standards
-
YAML Projector - Structured data projections
- Entity schemas
- Workflow definitions
- Integration mappings
-
Domain Entities, Processes, Features
- Canonical entity definitions
- Business process flows
- Feature-to-capability mappings
-
Task Graph
- Dependency-ordered task list
- Blocking relationships
- Completion criteria
-
Completeness Report
- Coverage analysis
- Gap identification
- Readiness score
User Story Generation
On entry to SPECIFICATION phase, the system triggers the user story task chain:
backlog_draft- Generate epics and stories from features and discovery databacklog_gap_scan- Identify coverage gaps (parallel)story_surface_mapper- Map stories to design surfaces (parallel)surface_block_mapper- Map surfaces to UI blocksspec_coverage_validator- Validate coverage before exportbacklog_diff- Compute version diffs (on-demand)
Stories follow standard format: as_a / i_want / so_that, with acceptance criteria, priority (MoSCoW), and traceability to features, surfaces, and blocks.
Finish-Line Convergence
Final validation before export completes:
- All user story tasks in terminal state
- Coverage validator confirms no critical gaps
- Readiness checker returns
ready=true - Export artifacts successfully generated
Two Question Systems
Critical Distinction
Praetor uses two completely separate question systems that must not be confused:
| Aspect | Discovery Questions | Plan Questions |
|---|---|---|
| Table | graph_nodes | plan_questions |
| Count | ~15 fixed per template | 50-150+ dynamic |
| Created | /api/flow/start | Transition to PLAN_GENERATION |
| Source | Template-driven | AI-generated per feature |
| Structure | Hierarchical graph | Flat list with grouping |
| Purpose | Understand WHAT to build | Define HOW to build |
Discovery Questions (graph_nodes)
// Structure in database
interface GraphNode {
id: string;
project_id: string;
type: 'section' | 'question' | 'suggestion';
parent_id: string | null;
content: {
question_text: string;
variants: QuestionVariant[];
answer_type: 'text' | 'select' | 'multi_select' | 'boolean';
};
status: 'pending' | 'answered' | 'skipped';
display_order: number;
}Flow:
- Template selected based on project type
template-instantiator.tscreates graph nodes- First question made active immediately
- Subsequent questions queued
- Answer triggers post-answer processing (suggestions, prefills)
Plan Questions (plan_questions)
// Structure in database
interface PlanQuestion {
id: string;
project_id: string;
question_plan_id: string;
feature_id: string | null;
question_text: string;
description: string;
answer_type: string;
options: Option[] | null;
prefilled_answer: string | null;
confidence_score: number;
user_answer: string | null;
status: 'pending' | 'prefilled' | 'answered' | 'skipped';
group_key: string;
display_order: number;
}Generation Process:
- Selected features analyzed
- AI generates questions per feature
- Questions grouped by feature/concern
- Prefill pass 1: Initial answer generation
- Prefill pass 2: Cross-reference refinement
- Low-confidence items → human tasks
Question Generation Technical Details
Question Sources Overview
Praetor has three distinct question sources, two active and one unused:
| Source | Status | Database Tables | Primary Use |
|---|---|---|---|
| Pool-Based Questions | ✅ ACTIVE | question_pools, pool_questions | Feature-driven plan generation |
| Template-Based Questions | ✅ ACTIVE | ontology_domain_templates, ontology_pattern_templates | QA loop, domain knowledge |
| Knowledge Base Questions | ❌ NOT USED | canonical_questions, entity_questions | Populated but never queried |
1. Pool-Based Questions (Primary Source)
Database Schema:
interface QuestionPool {
id: string;
name: string;
category: string;
industry?: string[];
scenario_type?: string;
created_at: Date;
updated_at: Date;
}
interface PoolQuestion {
id: string;
pool_id: string;
base_prompt: string;
question_type: string;
priority: number;
display_order: number;
prerequisite_questions?: string[];
metadata: Record<string, any>;
embedding?: number[]; // vector(1536) for semantic search
created_at: Date;
}
interface PoolActivationRule {
id: string;
pool_id: string;
trigger_type: 'keyword' | 'pattern' | 'domain' | 'entity';
trigger_value: string;
confidence_threshold: number;
industry_filter?: string[];
created_at: Date;
}Loader Service:
// src/services/flow/db-question-pool-loader.ts
export class DBQuestionPoolLoader {
constructor(private db: Pool) {}
// Load all pools
async getAllPools(): Promise<QuestionPool[]>
// Get questions by pool IDs
async getPoolQuestions(poolIds: string[]): Promise<PoolQuestion[]>
// Get activation rules
async getActivationRules(poolId: string): Promise<PoolActivationRule[]>
// Fallback to JSON files if DB is empty
private async loadFromJSON(): Promise<QuestionPool[]>
}JSON Fallback Location: /src/data/json/question-pools/ (40+ pool files)
2. Template-Based Questions (Secondary Source)
Database Schema:
interface DomainTemplate {
id: string;
domain: 'ecommerce' | 'saas' | 'healthcare' | 'internal-tools' | 'general';
category: 'entity' | 'workflow' | 'integration';
canonicalName: string;
displayName: string;
template: {
suggestedAttributes: Array<{
name: string;
type: string;
required: boolean;
description: string;
}>;
statusEnum?: string[];
transitions?: Array<{
from: string;
to: string;
trigger: string;
conditions?: string[];
}>;
businessRules?: string[];
requiredQuestions: string[]; // ← Questions embedded here
relatedEntities?: string[];
};
keywords: string[];
embedding?: number[];
created_at: Date;
}Service Classes:
// src/core/ontology/domain-template-service.ts (lines 23-89)
export class DomainTemplateService {
async getDomainKnowledge(
projectDescription: string,
existingAnswers: Record<string, any>
): Promise<DomainKnowledge> {
// Returns aggregated knowledge from templates
// Includes: entityTemplates, patternTemplates, integrationTemplates
// Extracts all requiredQuestions from matched templates
}
}3. Knowledge Base Questions (UNUSED)
Database Schema:
interface CanonicalQuestion {
id: string;
entity_id: string;
question_text: string;
category: string;
priority: number;
created_at: Date;
}
interface EntityQuestion {
id: string;
entity_id: string;
question_text: string;
question_type: string;
context: Record<string, any>;
created_at: Date;
}❌ NO USAGE FOUND:
- Population Scripts:
scripts/populate-question-pools.ts,scripts/generate-entity-questions.ts - Zero References: No active code queries these tables
- Intended Purpose: Unclear from existing code
⚠️ Partial KB Usage:
kb_naics_standard_mappings: Used ingetEntityPoolQuestionsForIndustry()(feature-driven-plan-generator.ts:195-319)kb_canonical_entities: Used for entity synonym matching- Other KB tables (
kb_capabilities,kb_patterns, etc.) are also unused
Active Question Generators
1. Feature-Driven Plan Generator ⭐ PRIMARY
File: src/services/flow/feature-driven-plan-generator.ts (689 lines)
Class Definition:
export class FeatureDrivenPlanGenerator {
constructor(
private db: Pool,
private poolLoader: DBQuestionPoolLoader,
private answerPrefiller: AnswerPrefillerService,
private questionTaskGenerator: QuestionTaskGenerator
) {}
async generatePlan(params: {
projectId: string;
curationId: string;
agentId: string;
}): Promise<FeatureDrivenPlan>
}Workflow (lines 42-148):
1. Load feature curation + selected features
2. Build feature structure:
├─ Query pool_questions by feature.relatedPoolIds (lines 357-371)
├─ Get entity pool questions via NAICS mapping (lines 195-319)
└─ Deduplicate questions
3. Create feature_driven_plans record
4. Create plan_sections (one per feature)
5. Insert plan_questions for each section
6. Prefill answers (two-pass AI process)
7. Generate tasks (REVIEW vs PROVIDE)
8. Return FeatureDrivenPlanKey Methods:
Lines 195-319: getEntityPoolQuestionsForIndustry()
// This is where KB tables ARE used
private async getEntityPoolQuestionsForIndustry(
industry: string,
projectId: string
): Promise<PoolQuestion[]> {
// 1. Map industry code → NAICS codes
// 2. Query kb_naics_standard_mappings for entity_ids
// 3. Get entity names from kb_canonical_entities
// 4. Find pool questions matching those entities
// Returns pool questions relevant to the industry
}Lines 357-371: Load Questions by Pool IDs
const poolQuestions = await this.poolLoader.getPoolQuestions(
feature.relatedPoolIds || []
);Database Operations:
- Reads:
feature_curations,selected_features,question_pools,pool_questions,kb_naics_standard_mappings,kb_canonical_entities - Writes:
feature_driven_plans,plan_sections,plan_questions
API Entry Point: POST /api/features/:projectId/curation
2. Planning Question Generator 🔄 SECONDARY
File: src/services/planning/planning-question-generator.ts (803 lines)
Class Definition:
export class PlanningQuestionGenerator {
constructor(
private db: Pool,
private poolLoader: DBQuestionPoolLoader,
private answerPrefiller: AnswerPrefillerService
) {}
async generatePlan(params: {
projectId: string;
blueprint: DomainBlueprint;
agentId: string;
}): Promise<PlanningQuestionPlan>
}Workflow (lines 178-277):
1. Analyze blueprint gaps:
├─ Missing entities?
├─ Missing workflows?
├─ Missing business rules?
├─ Missing compliance requirements?
├─ Missing integrations?
└─ Missing UX differentiation?
2. Select pools based on gaps (use PLANNING_POOL_IDS)
3. Load questions from selected pools (lines 545-566)
4. Filter to gap-specific question IDs
5. Create sections + questions
6. Prefill with domain context
7. Convert to DomainOpenQuestion for trackingPool Constants:
const PLANNING_POOL_IDS = {
ENTITY_DEFINITION: 'uuid-here',
WORKFLOW_DEFINITION: 'uuid-here',
BUSINESS_RULES: 'uuid-here',
COMPLIANCE: 'uuid-here',
INTEGRATIONS: 'uuid-here',
UX_DIFFERENTIATION: 'uuid-here'
};Usage: Planning phase, domain blueprint refinement
3. Design Question Generator 🎨 VISUAL DISCOVERY
File: src/services/design/design-question-generator.ts (534 lines)
Function Signature:
export async function generateDesignQuestionPlan(
db: Pool,
projectId: string,
blockCatalog: BlockCatalogService
): Promise<DesignQuestionPlan>Workflow (lines 263-390):
1. Load block catalog (NOT pool_questions!)
2. Group surfaces by section:
├─ Theme (color, typography, brand)
├─ Auth (login, signup, password reset)
├─ Marketing (landing, pricing, about)
├─ Navigation (header, footer, sidebar)
├─ Dashboard (analytics, tables, charts)
└─ Other surfaces (commerce, admin, settings, etc.)
3. For each surface:
├─ Generate hardcoded prompt (lines 202-245)
├─ Get matching blocks from catalog
└─ Build block selection options
4. Create DesignQuestionPlan structure
5. Store in projects.design_question_plan (JSONB column)Data Source: Block catalog service (separate from question pools)
Output Storage: Direct JSONB in projects table (not plan_questions)
4. Follow-up Generator 💡 REFINEMENT
File: src/services/flow/followup-generator.ts
Function Signature:
export async function generateFollowUpQuestions(
db: Pool,
projectId: string,
previousAnswers: Record<string, any>,
domainTemplateService: DomainTemplateService
): Promise<FollowUpQuestion[]>Workflow:
1. Get domain knowledge from templates
└─ domainTemplateService.getDomainKnowledge()
2. Build AI prompt with:
├─ Previous answers
├─ Entity templates (state machines)
├─ Integration capabilities
└─ Template requiredQuestions
3. Call AI to generate contextual follow-ups
4. Parse and structure questions
5. Return with context and priority5. QA Loop Orchestrator 🔁 SESSION MANAGEMENT
File: src/core/orchestration/qa-loop-orchestrator.ts
Three-Layer Question Strategy (lines 486-534):
async generateNextQuestions(session: QASession): Promise<Question[]> {
const questions: Question[] = [];
const askedQuestions = new Set<string>(session.askedQuestions || []);
// Layer 1: AI Evaluator Completeness Questions
const evaluatorQuestions = await this.completenessEvaluator.evaluate(session);
for (const q of evaluatorQuestions) {
if (!askedQuestions.has(q.question) && questions.length < count) {
questions.push(q);
askedQuestions.add(q.question);
}
}
// Layer 2: Domain Template Required Questions
const templateQuestions = session.domainKnowledge?.allRequiredQuestions || [];
for (const q of templateQuestions) {
if (!askedQuestions.has(q) && questions.length < count) {
questions.push({ question: q, source: 'template' });
askedQuestions.add(q);
}
}
// Layer 3: Semantic Retrieval from Templates
if (questions.length < count) {
const semanticQuestions = await this.semanticRetrieval.findRelevant(
session.currentContext,
count - questions.length
);
for (const q of semanticQuestions) {
if (!askedQuestions.has(q.question)) {
questions.push(q);
askedQuestions.add(q.question);
}
}
}
return questions;
}⚠️ Deduplication Issue: Only uses exact string matching (askedQuestions.has(q.question)). This misses paraphrased duplicates.
Deprecated & Dead Code
1. Question Plan Generator ❌ DEAD CODE
File: src/services/flow/question-plan-generator.ts (486 lines)
Status: Replaced by FeatureDrivenPlanGenerator
Original Purpose: Generic question plan before feature-driven approach
Why Deprecated:
- Uses hardcoded pool selection instead of feature-based
- Falls back to legacy activation rules
- Does not integrate with feature curation system
Still Referenced In:
phase-orchestrator.ts(import but not called)- Type definitions (interfaces still in use)
Removal Safety: Can be safely deleted (no active call sites)
2. Legacy Activation Rules ⚠️ FALLBACK ONLY
File: src/services/flow/pool-activation-rules.ts
Status: Only used if DB is empty
Current System: DBPoolActivationRules (database-backed)
Hardcoded Rules:
export const LEGACY_RULES: ActivationRule[] = [
{
poolId: 'auth-questions',
triggers: ['authentication', 'login', 'signup'],
confidence: 0.9
},
// ... more hardcoded rules
];Migration Path: Move remaining rules to pool_activation_rules table
3. JSON Question Pool Client ⚠️ FALLBACK ONLY
File: src/services/flow/question-pool-client.ts
Status: Only used if DB is empty
Current System: DBQuestionPoolLoader (database-backed)
JSON Files: /src/data/json/question-pools/*.json
Hot Reload: Dev mode file watching enabled
Answer Prefilling System
Two-Pass Refinement Process
File: src/services/flow/answer-prefill.ts (1,366 lines)
Main Workflow (lines 135-229):
export class AnswerPrefillerService {
async prefillAnswers(
questions: PlanQuestion[],
context: PrefillContext
): Promise<PrefilledAnswer[]> {
// Pass 1: Generate initial drafts (lines 478-703)
const drafts = await this.generateDrafts(questions, context);
// Pass 2: Refine with cross-references (lines 707-845)
const refined = await this.refineDrafts(drafts, context);
// Detect consistency issues (lines 1101-1134)
await this.detectInconsistencies(refined);
return refined;
}
}Pass 1: Draft Generation with Dependency Waves (lines 478-703)
Strategy: Process questions in waves based on dependencies
// Group questions by dependency level
const waves = this.buildDependencyWaves(questions);
// Process each wave in parallel
for (const wave of waves) {
const waveResults = await Promise.all(
wave.map(q => this.generateSingleDraft(q, context))
);
// Results from this wave inform next wave
}Confidence Scoring:
interface PrefilledAnswer {
questionId: string;
value: string;
confidence: number; // 0.0 - 1.0
reasoning: string;
sources: string[];
}Pass 2: Refinement (lines 707-845)
Cross-Reference Analysis:
async refineDrafts(
drafts: PrefilledAnswer[],
context: PrefillContext
): Promise<PrefilledAnswer[]> {
// For each draft:
// 1. Find related answers
// 2. Check for conflicts
// 3. Enhance with additional context
// 4. Adjust confidence based on consistency
// 5. Add validation notes
}Confidence Thresholds (lines 1154-1158)
const status = answer.confidence >= 0.9 ? 'auto_completed'
: answer.confidence >= 0.7 ? 'confirmed'
: 'pending';
// Workflow:
// confidence >= 0.9: Auto-accept, no user review needed
// 0.7 <= confidence < 0.9: Show to user as "confirmed", easy to accept
// confidence < 0.7: Mark as "pending", requires user reviewConsistency Detection (lines 1101-1134)
Cross-Answer Validation:
async detectInconsistencies(
answers: PrefilledAnswer[]
): Promise<InconsistencyReport[]> {
const inconsistencies: InconsistencyReport[] = [];
// Check for:
// - Contradicting values across related questions
// - Missing expected relationships
// - Type mismatches
// - Business rule violations
return inconsistencies;
}Task Generation System
Question-Task Integration
File: src/services/tasks/question-task-generator.ts (317 lines)
Main Workflow (lines 64-127):
export class QuestionTaskGenerator {
async generateTasksForPlan(
planId: string,
questions: PlanQuestion[]
): Promise<Task[]> {
// 1. Get questions needing REVIEW (confidence >= 0.5)
const reviewQuestions = await this.getReviewQuestions(questions);
// 2. Get questions needing PROVIDE (confidence < 0.5)
const provideQuestions = await this.getProvideQuestions(questions);
// 3. Build tasks
const tasks = [
...this.buildReviewTasks(reviewQuestions),
...this.buildProvideTasks(provideQuestions)
];
// 4. Insert into user_tasks table
await this.insertTasks(tasks);
return tasks;
}
}REVIEW vs PROVIDE Tasks
REVIEW Tasks (lines 133-152):
// For questions with confidence >= 0.5
// User reviews AI-generated answer
async getReviewQuestions(questions: PlanQuestion[]): Promise<PlanQuestion[]> {
return questions.filter(q =>
q.prefilled_answer &&
q.confidence_score >= 0.5 &&
q.status === 'prefilled'
);
}PROVIDE Tasks (lines 158-176):
// For questions with confidence < 0.5
// User provides answer from scratch
async getProvideQuestions(questions: PlanQuestion[]): Promise<PlanQuestion[]> {
return questions.filter(q =>
!q.prefilled_answer ||
q.confidence_score < 0.5 ||
q.status === 'pending'
);
}Task Input Structure (lines 192-299)
interface TaskInput {
type: 'REVIEW_ANSWER' | 'PROVIDE_ANSWER';
title: string;
description: string;
priority: number;
metadata: {
questionId: string;
questionText: string;
prefilledAnswer?: string;
confidence?: number;
reasoning?: string;
};
binding: {
targetType: 'plan_question';
targetId: string;
fieldPath: 'user_answer';
};
}Bidirectional Synchronization
File: src/services/tasks/question-task-sync-service.ts (305 lines)
Question → Task Sync (lines 48-96):
async onQuestionCompleted(questionId: string): Promise<void> {
// 1. Find linked task
const task = await this.findTaskByQuestionId(questionId);
// 2. Mark task as complete
await this.completeTask(task.id);
// 3. Copy answer to task output
await this.updateTaskOutput(task.id, answer);
}Task → Question Sync (lines 102-161):
async onTaskCompleted(taskId: string): Promise<void> {
// 1. Get task with binding
const task = await this.getTaskWithBinding(taskId);
// 2. Extract answer from task output
const answer = task.output.answer;
// 3. Update linked question
await this.updateQuestion(task.metadata.questionId, {
user_answer: answer,
status: 'answered'
});
// 4. Trigger post-answer processing
await this.postAnswerProcessor.process(questionId);
}Answer Analysis System
File: src/services/flow/answer-analyzer.ts (460 lines)
Main Analysis Workflow (lines 175-206)
export class AnswerAnalyzerService {
async analyze(
question: PlanQuestion,
answer: string,
context: AnalysisContext
): Promise<AnalysisResult> {
// 1. Keyword detection
const keywords = await this.detectKeywords(answer);
// 2. Entity extraction
const entities = await this.extractEntities(answer);
// 3. Ambiguity scoring
const ambiguityScore = await this.calculateAmbiguity(answer);
// 4. Follow-up decision
const needsFollowup = await this.determineFollowup(
question,
answer,
ambiguityScore
);
return { keywords, entities, ambiguityScore, needsFollowup };
}
}Keyword Detection (lines 235-260)
9 Categories:
const KEYWORD_CATEGORIES = {
authentication: ['login', 'signup', 'oauth', 'jwt', 'session'],
payment: ['stripe', 'checkout', 'payment', 'billing'],
notification: ['email', 'sms', 'push', 'notification'],
storage: ['s3', 'upload', 'file', 'media'],
search: ['search', 'filter', 'query', 'elasticsearch'],
analytics: ['tracking', 'analytics', 'metrics', 'events'],
admin: ['admin', 'dashboard', 'manage', 'crud'],
workflow: ['approval', 'workflow', 'process', 'state'],
integration: ['api', 'webhook', 'integration', 'sync']
};Ambiguity Calculation (lines 376-396)
function calculateAmbiguity(answer: string): number {
let score = 0;
// Signals of ambiguity:
if (answer.includes('maybe')) score += 0.2;
if (answer.includes('not sure')) score += 0.3;
if (answer.includes('probably')) score += 0.2;
if (answer.includes('I think')) score += 0.2;
if (answer.length < 20) score += 0.3; // Too short
if (!answer.includes('.')) score += 0.1; // No punctuation
return Math.min(score, 1.0);
}Follow-up Decision Logic (lines 402-450)
function determineFollowup(
question: PlanQuestion,
answer: string,
ambiguityScore: number
): boolean {
// Trigger follow-up if:
// 1. High ambiguity (> 0.6)
if (ambiguityScore > 0.6) return true;
// 2. Keywords detected but insufficient detail
const keywordCount = detectKeywords(answer).length;
if (keywordCount > 0 && answer.length < 50) return true;
// 3. Complex question with simple answer
if (question.complexity === 'high' && answer.length < 100) return true;
return false;
}Database Schema Reference
Question Pool Tables
CREATE TABLE question_pools (
id UUID PRIMARY KEY,
name VARCHAR(255) NOT NULL,
category VARCHAR(100),
industry VARCHAR(100)[],
scenario_type VARCHAR(100),
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE pool_questions (
id UUID PRIMARY KEY,
pool_id UUID REFERENCES question_pools(id),
base_prompt TEXT NOT NULL,
question_type VARCHAR(50),
priority INTEGER DEFAULT 0,
display_order INTEGER,
prerequisite_questions UUID[],
metadata JSONB,
embedding vector(1536),
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE pool_activation_rules (
id UUID PRIMARY KEY,
pool_id UUID REFERENCES question_pools(id),
trigger_type VARCHAR(50), -- 'keyword', 'pattern', 'domain', 'entity'
trigger_value TEXT,
confidence_threshold NUMERIC(3,2),
industry_filter VARCHAR(100)[],
created_at TIMESTAMPTZ DEFAULT NOW()
);Plan Storage Tables
CREATE TABLE feature_driven_plans (
id UUID PRIMARY KEY,
project_id UUID REFERENCES projects(id),
curation_id UUID REFERENCES feature_curations(id),
agent_id VARCHAR(255),
status VARCHAR(50) DEFAULT 'active',
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE plan_sections (
id UUID PRIMARY KEY,
plan_id UUID REFERENCES feature_driven_plans(id),
feature_id UUID REFERENCES feature_catalog(id),
title VARCHAR(255) NOT NULL,
description TEXT,
display_order INTEGER,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE plan_questions (
id UUID PRIMARY KEY,
section_id UUID REFERENCES plan_sections(id),
pool_question_id UUID REFERENCES pool_questions(id),
question_text TEXT NOT NULL,
display_order INTEGER,
answer TEXT,
prefill_value TEXT,
prefill_confidence NUMERIC(3,2),
auto_accepted BOOLEAN DEFAULT FALSE,
is_suppressed BOOLEAN DEFAULT FALSE,
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW()
);Domain Template Tables
CREATE TABLE ontology_domain_templates (
id UUID PRIMARY KEY,
domain VARCHAR(50) NOT NULL,
category VARCHAR(50) NOT NULL,
canonical_name VARCHAR(255) NOT NULL,
display_name VARCHAR(255),
template JSONB NOT NULL, -- Contains requiredQuestions[]
keywords TEXT[],
embedding vector(1536),
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE ontology_pattern_templates (
id UUID PRIMARY KEY,
pattern_name VARCHAR(255) NOT NULL,
display_name VARCHAR(255),
category VARCHAR(100),
complexity VARCHAR(50),
template JSONB NOT NULL, -- Contains requiredQuestions[]
keywords TEXT[],
compatible_domains TEXT[],
embedding vector(1536),
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE ontology_integration_templates (
id UUID PRIMARY KEY,
provider VARCHAR(255) NOT NULL,
display_name VARCHAR(255),
category VARCHAR(100),
capabilities TEXT[],
required_config TEXT[],
template JSONB NOT NULL, -- Contains webhookEvents, requiredQuestions
docs_reference TEXT,
embedding vector(1536),
created_at TIMESTAMPTZ DEFAULT NOW()
);Knowledge Base Tables (UNUSED)
CREATE TABLE canonical_questions (
id UUID PRIMARY KEY,
entity_id UUID,
question_text TEXT NOT NULL,
category VARCHAR(100),
priority INTEGER,
created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE TABLE entity_questions (
id UUID PRIMARY KEY,
entity_id UUID REFERENCES kb_canonical_entities(id),
question_text TEXT NOT NULL,
question_type VARCHAR(50),
context JSONB,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- ❌ These tables are populated but never queried in active codeKB Tables (Partial Use)
CREATE TABLE kb_naics_standard_mappings (
id UUID PRIMARY KEY,
naics_code VARCHAR(6) REFERENCES kb_naics_codes(code),
entity_id UUID REFERENCES kb_canonical_entities(id),
relevance_score NUMERIC(3,2),
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- ✅ Used in getEntityPoolQuestionsForIndustry()
CREATE TABLE kb_canonical_entities (
id UUID PRIMARY KEY,
entity_name VARCHAR(255) NOT NULL,
entity_type VARCHAR(100),
synonyms TEXT[],
description TEXT,
created_at TIMESTAMPTZ DEFAULT NOW()
);
-- ✅ Used for entity synonym matchingKnown Issues & Conflicts
1. Question Duplication Problem ⚠️ CRITICAL
Issue: Two systems (pool-based + template-based) may ask semantically identical questions
Example:
- Pool question: "What are your order status values?"
- Template question: "What are Order statuses and transitions?"
Root Cause: Only string-based deduplication in QA orchestrator
Current Mitigation (lines 486-534 of qa-loop-orchestrator.ts):
if (!askedQuestions.has(q.question) && questions.length < count) {
questions.push(q);
askedQuestions.add(q.question);
}Problem: Only catches exact string matches, misses:
- Paraphrasing ("What are Order statuses?" vs "Define order status values")
- Capitalization differences
- Semantic equivalence
Impact: Users see duplicate questions, confusion, wasted effort
2. Scope Overlap
Issue: Planning pools and domain templates cover same ground
Example:
PLANNING_POOL_IDS.ENTITY_DEFINITIONasks about entities- Domain entity templates also have
requiredQuestionsabout entities
When It Happens: Both systems active simultaneously
Impact: Double coverage, redundant work
3. No Priority Ordering
Issue: Pool questions have display_order, template questions don't
Problem: When merging both sources, unclear which to ask first
Current Behavior: Template questions appear in Layer 2/3 of QA loop, might interrupt pool flow
Impact: Suboptimal question ordering, broken dependency chains
4. Fragile Prerequisite Chains
Issue: Pool questions have prerequisite_questions, templates don't
Problem: Template question might be asked before its prerequisite from pool
Example:
- Pool Q1: "What entities do you need?" (prerequisite)
- Pool Q2: "What are Order attributes?" (depends on Q1)
- Template Q: "What are Order state transitions?" (no dependency tracking)
- Risk: Template Q asked before Q1, breaking logic flow
5. Unused KB Tables
Issue: canonical_questions, entity_questions populated but never queried
Wasted Effort: Population scripts run but data ignored
Questions:
- Should pool_questions be migrated to canonical_questions?
- Should the systems be merged?
- Should unused tables be deleted?
6. Planning Generator Bypasses Templates
Issue: PlanningQuestionGenerator doesn't use getDomainKnowledge()
Inconsistency: Feature-driven uses templates (via QA loop), planning doesn't
Result: Different question sources for similar concepts
Impact: Planning phase misses template enrichment
Conversation System Architecture
Graph-Based Conversation Structure
Primary Table: graph_nodes (discovery phase questions)
Hierarchical Structure:
interface GraphNode {
id: string;
project_id: string;
parent_id: string | null; // ← Creates tree structure
type: 'section' | 'question' | 'suggestion';
title: string;
description: string;
question_data: {
concept: string;
basePrompt: string;
formComponent: string;
options?: Array<{ value: string; label: string }>;
metadata?: Record<string, any>;
};
answer_data: {
value: string | string[] | number | boolean;
confidence?: number;
source?: 'user' | 'prefill' | 'agent';
} | null;
status: 'pending' | 'unanswered' | 'answered' | 'skipped';
order_index: number;
level: number; // Tree depth (0 = root)
is_dynamic: boolean; // Injected during conversation
trigger_conditions: Record<string, any> | null;
created_at: Date;
updated_at: Date;
answered_at: Date | null;
}Conversation Flow:
Section (type: 'section', level: 0)
├─ Question 1 (type: 'question', level: 1, parent_id: section.id)
│ ├─ Suggestion 1.1 (type: 'suggestion', level: 2, parent_id: q1.id)
│ └─ Dynamic Question 1.2 (injected based on answer, is_dynamic: true)
├─ Question 2 (type: 'question', level: 1)
└─ Question 3 (type: 'question', level: 1)Post-Answer Processing Pipeline
File: src/services/background/post-answer-processor.ts (863 lines)
Triggered When: User submits an answer to any question (graph_nodes or plan_questions)
Complete Workflow (lines 54-134):
async process(task: BackgroundTask): Promise<void> {
// Payload contains:
// - projectId, tenantId, userId, sessionId
// - questionNodeId (the answered question)
// - answer (the submitted answer value)
// STEP 1: Dynamic Question Generation (lines 88-283)
await this.checkAndCreateDynamicQuestions(payload, nodes)
// Sub-steps:
// 1. Calculate ontology completeness
// 2. Analyze completeness for gaps
// 3. Determine if question injection needed
// 4. Generate next best question
// 5. Inject into graph_nodes
// 6. Emit SSE event to frontend
// STEP 2: Follow-up Analysis (lines 91-486)
await this.analyzeAnswerAndGenerateFollowups(payload, nodes)
// Sub-steps:
// 1. Get answered question
// 2. Analyze answer (word count, complexity, confidence)
// 3. Generate follow-up questions using AI
// 4. Filter by confidence threshold (>= 0.75)
// 5. Check activated pools to avoid duplicates
// 6. Inject best follow-up
// 7. Emit SSE event
// STEP 3: Pool Activation (lines 97-591)
await this.checkAndActivatePools(payload, nodes)
// Sub-steps:
// 1. Analyze answer text for keywords
// 2. Match against pool activation rules
// 3. Get high-confidence activations
// 4. Check if pools already activated
// 5. Activate each new pool
// 6. Mark pool as activated
// 7. Emit SSE events
}Dynamic Question Injection (lines 747-836)
Three Mechanisms:
-
Gap-Filling (lines 189-283):
// Triggered by: Ontology incompleteness // Conditions: if (ontology.overallCompleteness < THRESHOLD) { const recommendation = analyzer.getNextBestQuestion(analysis) await this.injectDynamicQuestion({ questionText: recommendation.questionText, targetEntity: recommendation.targetEntity, targetAttribute: recommendation.targetAttribute, priority: recommendation.priority, rationale: recommendation.rationale }) } -
Follow-up Generation (lines 288-486):
// Triggered by: Ambiguous or incomplete answer // Conditions: if (analysis.requiresFollowup && analysis.confidence >= 0.6) { const followups = await generator.generate(...) const bestFollowup = generator.getBestFollowup( generator.filterByConfidence(followups, 0.75) ) await this.injectDynamicQuestion({ ... }) } -
Pool Activation (lines 491-701):
// Triggered by: Keywords in answer // Example: User mentions "Stripe" → activates payment-stripe pool const analysis = await poolRules.analyzeForActivation(answerText) for (const activation of analysis.highConfidenceActivations) { if (!activatedPools.includes(activation.poolId)) { await this.activatePool(activation, params, nodes) await this.markPoolAsActivated(projectId, poolId) } }
Injection Process:
// Insert new node into graph_nodes table
const nodeId = await db.insert({
type: 'question',
title: concept,
description: rationale,
question_data: {
concept,
basePrompt: questionText,
formComponent,
metadata: { targetEntity, targetAttribute, priority }
},
is_dynamic: true, // ← Marks as injected
trigger_conditions: { type: 'gap-filling', targetEntity },
order_index: maxOrder + 1, // Append to end
status: 'unanswered'
})Activated Pools Tracking
Table: activated_pools (created dynamically, lines 640-664)
CREATE TABLE IF NOT EXISTS activated_pools (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id TEXT NOT NULL,
project_id TEXT NOT NULL,
pool_id TEXT NOT NULL,
pool_name TEXT NOT NULL,
reasoning TEXT,
activated_at TIMESTAMPTZ DEFAULT NOW(),
UNIQUE(tenant_id, project_id, pool_id)
);Purpose:
- Prevent duplicate pool activations
- Track why each pool was activated
- Query existing activations before injecting pool questions
Task System Architecture
Task Types and Lifecycle
Primary Tables:
tasks- User-facing task recordstask_actions- Available actions on taskstask_targets- Resources linked to taskstask_activity- Audit log of task changes
Task States:
OPEN → IN_PROGRESS → DONE
↓ ↓ ↓
SKIPPED CANCELLED (terminal states)User Task Service
File: src/services/tasks/user-task-service.ts (806 lines)
Core Operations:
1. Task Creation (lines 22-68):
async createTask(input: CreateTaskInput): Promise<Task> {
// Calls database function create_task_with_actions()
// Creates task + actions + targets atomically
// Returns full Task object
}
async createTaskIfNotExists(input: CreateTaskInput): Promise<Task | null> {
// Deduplication logic:
// - Check if task exists for plan_question_id
// - Check if task exists for graph_node_id
// - Only create if no existing task
// Returns null if task already exists
}2. Question-Task Linking (lines 74-210):
// Tasks can link to questions via:
// - plan_question_id (plan-phase questions)
// - graph_node_id (discovery questions)
async taskExistsForPlanQuestion(planQuestionId: string): Promise<boolean>
async getTaskByPlanQuestion(planQuestionId: string): Promise<Task | null>
async cancelTasksForQuestion(
planQuestionId: string | null,
graphNodeId: string | null,
reason: string
): Promise<number>
// Used when question is skipped or removed3. Task Querying (lines 308-412):
async getTasksWithCount(filter: TaskFilter): Promise<TaskListResult> {
// Filtering:
// - status, taskType, category, priority
// - assignedTo, blocking flag
// - Text search on title/description
// - Exclude blocked tasks (depends_on not satisfied)
// Ordering:
// 1. By status (OPEN first, then IN_PROGRESS, then others)
// 2. By blocking flag (blocking tasks first)
// 3. By priority (urgent → high → medium → low)
// 4. By created_at (oldest first)
}
async getNextActionableTask(projectId: string, userId?: string): Promise<Task | null>
// Returns highest priority unblocked task4. Task State Transitions (lines 511-610):
async claimTask(taskId: string, userId: string): Promise<boolean>
// Atomic claim (sets assigned_to)
async startTask(taskId: string, userId?: string): Promise<void>
// OPEN → IN_PROGRESS, sets started_at
async completeTask(taskId: string, result: Record<string, unknown>, actor?: string): Promise<void>
// Calls complete_task_with_result() DB function
// Executes output bindings
// Updates question if linked
async skipTask(taskId: string, userId?: string, reason?: string): Promise<void>
// Only for non-blocking tasks
async cancelTask(taskId: string, userId?: string, reason?: string): Promise<void>
// Can cancel any task5. Task Statistics (lines 678-710):
async getStatistics(projectId: string): Promise<TaskStatistics> {
return {
totalTasks: number,
openTasks: number,
inProgressTasks: number,
completedTasks: number,
blockedTasks: number, // Has unsatisfied dependencies
blockingTasks: number // Marked as blocking
}
}Background Task System
File: src/services/background/worker.ts (788 lines)
Architecture:
BackgroundWorker (singleton)
├─ Poll Loop (every 2s)
├─ Task Handlers (Map<TaskType, Handler>)
├─ Concurrency Control (max 3 concurrent)
└─ Task Detectors (auto-trigger chains)Worker Configuration (lines 66-99):
export class BackgroundWorker {
private config: WorkerConfig
private isRunning: boolean = false
private pollTimer: NodeJS.Timeout | null = null
private activeTaskCount: number = 0
private handlers: Map<TaskType, TaskHandler> = new Map()
private detectorUnsubscribers: Array<() => void> = []
constructor(config: WorkerConfig) {
this.config = {
pollIntervalMs: config.pollIntervalMs || 2000, // Poll every 2s
maxConcurrent: config.maxConcurrent || 3, // Max 3 parallel tasks
taskTypes: config.taskTypes, // Optional filter
emitter: config.emitter, // SSE event emitter
ontologyTracker: config.ontologyTracker // Optional tracker
}
}
}Registered Task Types (lines 104-276):
- Post-Answer Processing (lines 114-116)
- Feature Curation (lines 119-126)
- Visual Discovery - Research (lines 132-175):
- market-research
- design-intelligence
- technical-feasibility
- competitive-analysis
- visual-curation
- architecture-analysis
- business-strategy
- product-management
- generic-research
- Visual Discovery - Option Generation (lines 181-209):
- color-palette-generation
- typography-generation
- layout-generation
- component-generation
- spacing-generation
- style-preference-generation
- Asset Processing (lines 215-228):
- asset-vectorization
- asset-analysis
- preference-extraction
- Flow Optimization (lines 234-242):
- enhance-variant
- generate-suggestions
- User Stories (lines 248-270):
- backlog_draft
- backlog_gap_scan
- story_surface_mapper
- surface_block_mapper
- backlog_diff
- spec_coverage_validator
Worker Poll Loop (lines 363-406):
private async poll(): Promise<void> {
if (!this.isRunning) return
// Check capacity
if (this.activeTaskCount >= this.config.maxConcurrent) {
this.scheduleNextPoll()
return
}
const taskService = getTaskService()
// Claim next task (FIFO within priority)
let task: BackgroundTask | null = null
if (this.config.taskTypes) {
// Try each type in order
for (const taskType of this.config.taskTypes) {
task = await taskService.claimNextTask(taskType)
if (task) break
}
} else {
// Claim any task type
task = await taskService.claimNextTask()
}
if (task) {
// Process in background (don't await)
this.processTask(task).catch(...)
// Immediately poll again if we have capacity
if (this.activeTaskCount < this.config.maxConcurrent) {
setImmediate(() => this.poll())
return
}
}
// Schedule next poll
this.scheduleNextPoll() // setTimeout(poll, pollIntervalMs)
}Task Processing (lines 420-522):
private async processTask(task: BackgroundTask): Promise<void> {
this.activeTaskCount++
const timer = createTimer()
try {
const handler = this.handlers.get(task.taskType)
if (!handler) {
await taskService.failTask(task.id, `No handler for: ${task.taskType}`)
return
}
logger.info({ taskId, taskType, projectId }, 'Processing task')
await handler(task) // Execute handler
logger.info({ taskId, durationMs: timer() }, 'Task completed')
// Log to orchestration_debug_events
await logOrchestrationEvent({
eventType: 'step.completed',
payload: { taskId, taskType, durationMs }
})
} catch (error) {
logger.error({ error, taskId }, 'Task failed')
await taskService.failTask(task.id, error.message, { stack: error.stack })
await logOrchestrationEvent({
eventType: 'step.failed',
payload: { taskId, error, durationMs }
})
} finally {
this.activeTaskCount--
}
}Task Detectors (lines 314-334):
// Auto-trigger task chains based on events
private initializeDetectors(): void {
// Option generation: Tier 2 tasks when Tier 1 completes
const optionUnsubscribe = initOptionGenerationDetector()
// User story: Triggers backlog generation chains
const userStoryUnsubscribe = initUserStoryDetector()
// Feature curation: External event-triggered
initFeatureCurationDetector()
}Example: Option Generation Detector
// When market-research task completes:
eventEmitter.on('task.completed', async (event) => {
if (event.taskType === 'market-research') {
// Auto-trigger color-palette-generation
await taskService.createTask({
taskType: 'color-palette-generation',
projectId: event.projectId,
parameters: { researchResults: event.result }
})
}
})E2E Test Utilities (lines 591-787)
Flush Tasks (lines 602-741):
export async function flushBackgroundTasks(
projectId: string,
options: {
taskTypes?: TaskType[]
maxTasks?: number // default: 100
continueOnError?: boolean // default: true
taskTimeoutMs?: number // default: 60000
}
): Promise<FlushTasksResult>
// Usage in tests:
const result = await flushBackgroundTasks(projectId, {
taskTypes: ['post-answer-processing'],
maxTasks: 10
})
// Result: { processed: 5, succeeded: 5, failed: 0, tasks: [...] }Wait for Tasks (lines 750-787):
export async function waitForBackgroundTasks(
projectId: string,
options: {
timeoutMs?: number // default: 300000 (5 min)
pollIntervalMs?: number // default: 1000
}
): Promise<{ completed: number; failed: number; timedOut: boolean }>
// Usage in tests:
await waitForBackgroundTasks(projectId, { timeoutMs: 60000 })User Stories System
Overview
The User Stories System bridges the gap between Visual Discovery and Specification phases, transforming gathered requirements and design choices into implementation-ready user stories. This system provides structured, traceable stories that connect features to design surfaces, UI blocks, and domain concepts.
Task Chain
User stories are generated through a sequential chain of background tasks, with automatic progression between steps:
-
backlog_draft - Generates epics and stories from selected features and discovery data
- Input: Feature selections, discovery output, planning artifacts
- Output: Initial backlog with epics and stories in
user_story_backlogstable - Story format:
as_a/i_want/so_that, acceptance criteria (JSONB), priority (MoSCoW) - Triggers: Features selected + Visual Discovery readiness >= 80%, OR SPECIFICATION phase entry
-
backlog_gap_scan - Identifies coverage gaps (runs in parallel with story_surface_mapper)
- Input: Generated stories, selected features
- Output: Coverage report identifying uncovered must-have features
- Runs parallel to:
story_surface_mapper - Critical gaps inform readiness scoring
-
story_surface_mapper - Maps stories to design surfaces (runs in parallel with backlog_gap_scan)
- Input: Stories from backlog_draft
- Output: Story-to-surface mappings in
user_story_tracestable - Runs parallel to:
backlog_gap_scan - Surfaces: Login page, dashboard, product list, checkout flow, etc.
-
surface_block_mapper - Maps surfaces to UI blocks (triggered after story_surface_mapper completes)
- Input: Surface mappings, block catalog
- Output: Surface-to-block mappings, component suggestions
- Depends on:
story_surface_mappercompletion - Uses block hierarchy and complexity levels from catalog
-
spec_coverage_validator - Validates coverage before export (triggered after surface_block_mapper)
- Input: Complete story set, traces, coverage reports
- Output: Validation report with coverage metrics
- Depends on:
surface_block_mappercompletion - Blocks export if critical gaps remain
-
backlog_diff - Computes version diffs (on-demand, not part of main chain)
- Input: Two backlog versions
- Output: Diff report showing added/modified/removed stories
- Triggered: When user requests comparison between backlog versions
Trigger Conditions
The user story task chain can be triggered in two ways:
-
Automatic Trigger (Visual Discovery): When features are selected AND Visual Discovery readiness >= 80%, the
backlog_drafttask is created. This allows early story generation while research tasks complete. -
Phase Entry Trigger (Specification): When the project transitions to SPECIFICATION phase, the task chain is forcibly triggered with readiness=100, ensuring all stories are generated before export.
Task Completion Handler: handleUserStoryTaskCompletion() (in user-story-detector.ts) automatically triggers the next task in the chain when the previous task reaches DONE status. This creates a fire-and-forget pipeline where task completion events drive progression.
Story Format
Each user story follows a structured format stored in the user_stories table:
interface UserStory {
id: string
backlog_id: string
epic_id?: string
// Story statement
as_a: string // User role (e.g., "customer", "admin")
i_want: string // Desired capability
so_that: string // Business value
// Acceptance criteria
acceptance_criteria: {
given: string
when: string
then: string
}[]
// Priority and effort
priority: 'must_have' | 'should_have' | 'could_have' | 'wont_have' // MoSCoW
estimated_effort: 'xs' | 's' | 'm' | 'l' | 'xl'
// Traceability
trace: {
surfaces: string[] // Design surfaces this story affects
blocks: string[] // UI blocks needed for implementation
concepts: string[] // Domain concepts referenced
features: string[] // Features this story implements
visualAssets: string[] // Design assets (colors, typography, etc.)
}
status: 'draft' | 'refined' | 'ready' | 'in_progress' | 'done'
}Database Tables
The system uses four primary tables:
-
user_story_backlogs - Backlog containers
- One backlog per project version
- Tracks overall backlog status and metadata
- Links to project_id and design_version_id
-
user_story_epics - Epic groupings
- High-level feature groupings
- Contains multiple related stories
- Priority, status, and progress tracking
-
user_stories - Individual stories
- Core story data (as_a/i_want/so_that)
- Acceptance criteria, priority, effort
- Status and assignee information
-
user_story_traces - Traceability links
- Story-to-surface mappings
- Story-to-block mappings
- Story-to-feature mappings
- Bidirectional navigation support
-
user_story_coverage_reports - Coverage analysis
- Identifies uncovered must-have features
- Tracks coverage percentages
- Highlights critical gaps
Coverage Validation
The spec_coverage_validator task performs comprehensive coverage analysis:
- Feature Coverage: All must-have features have associated stories
- Surface Coverage: All design surfaces have implementation stories
- Block Coverage: All selected blocks are referenced by stories
- Critical Gaps: Uncovered must-have features block export readiness
Coverage metrics inform the final readiness score in SPECIFICATION phase. Projects with critical gaps (uncovered must-have features) cannot proceed to export until gaps are resolved (either by generating additional stories or downgrading feature priority).
Complete Question-to-Task Flow
Discovery Phase Flow
User Answers Question
↓
[API: POST /api/flow/answer]
↓
1. Save answer to graph_nodes.answer_data
2. Create background task: 'post-answer-processing'
↓
Background Worker Claims Task
↓
PostAnswerProcessor.process()
↓
├─ Dynamic Question Injection
│ └─ New graph_node created (is_dynamic: true)
├─ Follow-up Analysis
│ └─ New graph_node created (follow-up question)
└─ Pool Activation
└─ New graph_node created (first pool question)
↓
SSE Events Emitted → Frontend UpdatesPlan Phase Flow
Feature-Driven Plan Generated
↓
plan_questions created with prefilled answers
↓
QuestionTaskGenerator.generateTasksForPlan()
↓
For each question:
if (confidence >= 0.5) → REVIEW task
if (confidence < 0.5) → PROVIDE task
↓
Tasks created with:
- plan_question_id link
- Task actions (APPROVE, EDIT, SKIP)
- Output bindings (to plan_questions.user_answer)
↓
User Completes Task
↓
Task completion triggers binding execution
↓
plan_questions.user_answer updated
plan_questions.status = 'answered'
↓
Task marked as DONEAdditional Issues Found
7. Post-Answer Duplicate Detection Issue
File: src/services/background/post-answer-processor.ts (lines 392-433)
Problem: Follow-up questions are filtered to avoid pool duplicates, but uses regex pattern matching
const poolTopicPatterns: Record<string, RegExp[]> = {
'payment-stripe': [
/stripe/i,
/payment.*method/i,
/payment.*feature/i,
// ...
]
}
// Filter out questions matching activated pool topics
highConfidenceQuestions = highConfidenceQuestions.filter((q) => {
const questionText = q.questionText.toLowerCase()
return !coveredPatterns.some((pattern) => pattern.test(questionText))
})Issue:
- Hardcoded pattern list (doesn't scale)
- Regex matching can miss paraphrased questions
- No semantic similarity check
- Pattern list must be manually maintained
8. Conversation History Context Issue
File: src/services/background/post-answer-processor.ts (lines 353-362)
Context Building:
const conversationHistory = nodes
.filter((n) => n.answerData)
.map((n) => ({
question: n.questionData?.concept || n.title,
answer: typeof n.answerData?.value === 'string'
? n.answerData.value
: JSON.stringify(n.answerData?.value)
}))Issue:
- Includes ALL answered questions (no limit)
- Could exceed AI context window for long conversations
- No truncation or summarization
- No recency weighting
9. Pool Activation Constraint Injection
File: src/services/background/post-answer-processor.ts (lines 523-531)
Context Augmentation:
const projectConstraints: string[] = projectMetadata?.constraints || []
let textToAnalyze = answerText
if (projectConstraints.length > 0) {
const constraintText = projectConstraints.join(' ')
textToAnalyze = `${answerText} [Project Constraints: ${constraintText}]`
}Clever but Fragile:
- Appends constraints to answer text for analysis
- Increases likelihood of constraint-related pool activation
- But: AI might treat bracketed text differently
- No explicit constraint-pool mapping
10. Dynamic Question Priority Conflict
Issue: Three injection mechanisms all add questions with different priorities:
- Gap-filling: Uses
recommendation.priority(variable) - Follow-up: Uses priority
9(lines 468, 684) - Pool activation: Uses priority
9(line 684)
Problem:
- No coordination between mechanisms
- Multiple questions could be injected simultaneously
- User sees multiple new questions at once
- Unclear which to answer first
Impact: Confusing UX when multiple mechanisms fire
11. Task Deduplication Only on IDs
File: src/services/tasks/user-task-service.ts (lines 74-169)
Deduplication Logic:
async createTaskIfNotExists(input: CreateTaskInput): Promise<Task | null> {
// Check for existing task by plan question ID
if (input.planQuestionId) {
const exists = await this.taskExistsForPlanQuestion(input.planQuestionId)
if (exists) return null
}
// Check for existing task by graph node ID
if (!input.planQuestionId && input.graphNodeId) {
const exists = await this.taskExistsForGraphNode(input.graphNodeId)
if (exists) return null
}
return this.createTask(input)
}Issue:
- Only checks exact ID matches
- Doesn't check semantic task similarity
- Could create duplicate tasks for same work if IDs differ
- No check for similar task titles/descriptions
12. Background Worker Concurrency Race
File: src/services/background/worker.ts (lines 368-399)
Race Condition:
if (this.activeTaskCount >= this.config.maxConcurrent) {
this.scheduleNextPoll()
return
}
// ... later ...
if (task) {
// Process in background (don't await!)
this.processTask(task).catch(...)
// Immediately poll again if we have capacity
if (this.activeTaskCount < this.config.maxConcurrent) {
setImmediate(() => this.poll())
return
}
}Potential Issue:
activeTaskCountincremented insideprocessTask()(line 421)- But
processTask()is called without await - Brief window where
activeTaskCounthasn't incremented yet - Could claim more tasks than
maxConcurrentallows
Likelihood: Low (small time window), but possible under high load
Data Flow
From User Input to Specification Output
┌─────────────────────────────────────────────────────────────────────────────┐
│ USER INPUT FLOW │
│ │
│ User ──▶ Question ──▶ Answer ──▶ Post-Processing ──▶ Next Question │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ graph_nodes answers ┌─────────────────┐ │
│ or │ Post-Answer │ │
│ plan_questions │ Processing: │ │
│ │ - Ontology update│ │
│ │ - Suggestions │ │
│ │ - Prefill cascade│ │
│ │ - Task detection │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ SPECIFICATION COMPILATION │
│ │
│ Discovery Output ──▶ Feature Selection ──▶ Plan Answers ──▶ Spec Compiler │
│ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Entity │ │ Feature │ │ Require- │ │ Multi- │ │
│ │Extraction│ │ Context │ │ ments │ │ Agent │ │
│ │ │ │ │ │ │ │ Enhance │ │
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
│ │ │ │ │ │
│ └────────────────────┴───────────────────┴────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ SpecV1 │ │
│ │ (Validated) │ │
│ │ │ │
│ │ - Entities │ │
│ │ - Endpoints │ │
│ │ - Workflows │ │
│ │ - Requirements │ │
│ │ - Test Cases │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘Post-Answer Processing Pipeline
When a user answers a question, the following pipeline executes:
// Simplified flow from post-answer-processor.ts
async function processAnswer(answer: Answer) {
// 1. Update ontology with extracted entities
await ontologyTracker.extractAndUpdate(answer);
// 2. Generate follow-up suggestions
const suggestions = await suggestionGenerator.generate(answer);
// 3. Cascade prefills to related questions
await prefillCascade.propagate(answer);
// 4. Detect new tasks
await taskDetector.checkForTasks(answer);
// 5. Update completion metrics
await completionManager.recalculate(answer.project_id);
// 6. Check phase readiness
await phaseOrchestrator.checkReadiness(answer.project_id);
}Knowledge Base Architecture
The knowledge base is Praetor's foundation for intelligent feature matching and requirement understanding. It consists of three layers: standardized entities, architectural patterns, and generated features.
Layer 1: Standardized Entities (3,000+ entities)
Industry-specific vocabularies imported from authoritative sources:
Universal Standards:
- Schema.org (~827 types): Universal vocabulary for things, actions, and creative works
- NAICS (~1,012 codes): North American Industry Classification System
Domain-Specific Standards:
| Standard | Entities | Domain | Import Script |
|---|---|---|---|
| FHIR R4 | 157 | Healthcare | import-fhir-r4.ts |
| HR Open | 50 | HR/Workforce | import-hr-open.ts |
| Ed-Fi | 86 | Education | import-ed-fi.ts |
| SALI | 267 | Legal | import-sali.ts |
| ISO 20022 | 323 | Finance | import-iso20022.ts |
| RESO | 331 | Real Estate | import-reso.ts |
Storage: kb_standard_entities table with:
standard_code: 'FHIR_R4', 'HR_OPEN', etc.entity_name: 'Patient', 'JobPosting', etc.properties: JSONB array of field definitionsmetadata: Standard-specific data
Layer 2: Architectural Patterns (751 patterns)
Design patterns and best practices from security, infrastructure, and cloud frameworks:
Cloud-Agnostic Patterns (642 patterns):
| Source | Count | Focus | Cloud Provider |
|---|---|---|---|
| MITRE ATT&CK | 389 | Security threats & mitigations | agnostic |
| OWASP ASVS | 166 | Application security | agnostic |
| NIST OSCAL | 87 | Compliance & controls | agnostic |
| CNCF Landscape | ~240 | Cloud-native infrastructure | agnostic |
| 12-Factor App | ~42 (planned) | Cloud-native app design | agnostic |
Cloud-Specific Patterns (109+ patterns):
| Source | Count | Focus | Cloud Provider |
|---|---|---|---|
| AWS Well-Architected | 109 | AWS best practices | aws |
| GCP Architecture Framework | ~285 (planned) | GCP best practices | gcp |
| Azure Well-Architected | TBD | Azure best practices | azure |
Storage: kb_patterns table with:
cloud_provider: 'agnostic', 'aws', 'gcp', 'azure'category: Pattern groupingdescription: What the pattern achievesmetadata: Pattern-specific data (services, when to use, antipatterns)
Cloud Provider Selection Logic:
// User selects 'gcp' → load patterns WHERE cloud_provider IN ('agnostic', 'gcp')
// User selects 'agnostic' → load patterns WHERE cloud_provider = 'agnostic'
// Prefer cloud-specific over agnostic when both existLayer 3: Generated Features (165 features)
User-facing capabilities AI-generated from patterns:
Feature Catalog (Migration 095 Enhancement):
- 165 features total (77 new, 13 updated in February 2026)
- 85 commodity features (authentication, database access, etc.)
- 80 variable features (domain-specific capabilities)
- 9 categories
Feature → Pattern Linkages:
related_patternscolumn: Array of pattern IDs (TEXT[])kb_pattern_feature_mappingsjunction table: 129 verified mappings- Each feature references 1-N patterns that informed its creation
- Each pattern can inform multiple features
Feature → Pool Linkages:
related_pool_idscolumn: Array of question pool IDs- 90/165 features have question pools attached
- Links features to detailed questions for specification
Feature Metadata:
interface Feature {
id: string
name: string // "User Authentication", "Payment Processing"
description: string
category: string // "Identity & Access", "Commerce"
// Matching metadata
applies_to: {
industries: string[] // ["all"], ["healthcare", "finance"]
project_types: string[] // ["saas", "marketplace"]
}
supports_goals: string[] // ["security", "compliance", "user-management"]
// Linkages
related_patterns: string[] // Pattern IDs from kb_patterns
related_pool_ids: string[] // Question pool IDs
// Cloud awareness
cloud_provider?: string // Inherits from primary pattern
}Data Flow: Standards → Patterns → Features → Questions
┌─────────────────────────────────────────────────────────────────┐
│ KNOWLEDGE BASE LAYER │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Standards │ → │ Patterns │ → │ Features │ │
│ │ │ │ │ │ │ │
│ │ Schema.org │ │ MITRE ATT&CK │ │ 165 features │ │
│ │ NAICS │ │ OWASP ASVS │ │ Commodity: 85│ │
│ │ FHIR │ │ NIST │ │ Variable: 80 │ │
│ │ HR Open │ │ CNCF │ │ 9 categories │ │
│ │ Ed-Fi │ │ 12-Factor │ │ │ │
│ │ SALI │ │ AWS (109) │ │ With cloud- │ │
│ │ ISO 20022 │ │ GCP (plan) │ │ awareness │ │
│ │ RESO │ │ │ │ │ │
│ │ │ │ │ │ │ │
│ │ 3,000+ │ │ 751 patterns │ │ 129 pattern │ │
│ │ entities │ │ 642 agnostic │ │ mappings │ │
│ │ │ │ 109 AWS │ │ │ │
│ └──────────────┘ └──────────────┘ └──────┬───────┘ │
│ │ │
└───────────────────────────────────────────────────┼─────────────┘
│
┌────────────────▼─────────────┐
│ RUNTIME LAYER │
│ │
│ Discovery → Feature Match │
│ Feature → Question Pools │
│ Questions → Specifications │
└──────────────────────────────┘Feature Matching Pipeline (Enhanced with Structured Context)
Before Phase Fix (keyword matching):
// Old approach: naive keyword search
const signals = extractKeywords(answers); // ["payment", "stripe", "checkout"]
const features = featureCatalog.filter(f =>
signals.some(s => f.name.toLowerCase().includes(s))
);After Phase Fix (structured signal extraction):
// 1. Bridge requirements discovery to DiscoveryContext
const context = await bridgeRequirementsDiscovery(projectId, tenantId);
// 2. Extract structured signals
const signals = {
goals: context.purpose_and_problem.goals, // ["accept payments", "subscription billing"]
painPoints: context.users_and_personas.painPoints, // ["manual invoicing", "payment failures"]
jobsToBeDone: context.users_and_personas.jobsToBeDone, // ["process credit cards"]
constraints: context.constraints.existingSystems, // ["Stripe"]
dealbreakers: context.success_criteria.dealbreakers, // ["PCI compliance required"]
sensitiveData: context.non_functional_screening.sensitiveData, // ["credit cards"]
expectedScale: context.non_functional_screening.expectedScale // ["10k transactions/month"]
};
// 3. Match features using structured signals
const features = featureMatcher.matchFeatures(context);
// Returns: Payment Processing, Stripe Integration, PCI Compliance, Subscription Management
// 4. Score and rank by relevance
const rankedFeatures = scoreFeatures(features, context);Scoring Algorithm:
function scoreFeature(feature: Feature, context: DiscoveryContext): number {
let score = 0;
// Industry match (30 points)
if (feature.applies_to.industries.includes(context.industry)) score += 30;
else if (feature.applies_to.industries.includes('all')) score += 10;
// Goal match (20 points per match)
const goalMatches = context.goals.filter(g => feature.supports_goals.includes(g));
score += goalMatches.length * 20;
// Constraint match (25 points per match)
const constraintMatches = context.constraints.mustHaves?.filter(c =>
feature.supports_goals.includes(c.toLowerCase())
) || [];
score += constraintMatches.length * 25;
// Existing system integration (15 points)
if (context.constraints.existingSystems?.length > 0) {
if (feature.category === 'Integrations') score += 15;
}
// Pattern match (10 points per pattern)
const patternMatches = feature.related_patterns.filter(p =>
detectedPatterns.includes(p)
);
score += patternMatches.length * 10;
return score;
}Population Scripts (Offline Generation)
1. Import Standards (one-time):
pnpm run import:schema-org # ~827 types
pnpm run import:naics # ~1,012 codes
pnpm run import:fhir # ~157 resources
pnpm run import:hr-open # 50 entities
pnpm run import:ed-fi # 86 entities
pnpm run import:sali # 267 entities
pnpm run import:iso20022 # 323 entities
pnpm run import:reso # 331 entities2. Import Patterns (one-time):
pnpm run import:mitre-attack # ~389 patterns
pnpm run import:owasp-asvs # ~166 patterns
pnpm run import:nist-oscal # ~87 patterns
pnpm run import:cncf-landscape # ~240 patterns
pnpm run import:12-factor # ~42 patterns (planned)
pnpm run import:aws-patterns # 109 patterns
pnpm run import:google-cloud # ~285 patterns (planned)3. Generate Features (run after pattern updates):
pnpm run populate:feature-catalog
# - Reads kb_patterns grouped by category
# - Uses AI to generate user-facing features from pattern groups
# - Links features to patterns via related_patterns[]
# - Links features to question pools via related_pool_ids[]
# - Populates kb_pattern_feature_mappings junction table
# - Output: 165 features with full metadata4. Verify Linkages:
pnpm run verify:linkages
# Checks:
# - All features have related_patterns populated
# - All features have related_pool_ids populated
# - All pattern IDs in related_patterns exist in kb_patterns
# - All feature IDs in junction table exist in feature_catalog
# - No orphaned patterns (all patterns referenced by at least one feature)Runtime Usage
Feature Selection (Phase 2):
- Load
requirements_discovery_contextsfor project - Bridge to
DiscoveryContextinterface - Extract structured signals (goals, pain points, constraints, etc.)
- Load
feature_seedsfor prioritization hints - Match features using scoring algorithm
- Rank by relevance score
- Return top N features to user
Question Pool Activation (Phase 1 & 3):
- During discovery: Monitor user messages for technology mentions
- Check
DatabasePoolActivationRulesfor matches - If high-confidence match (>0.8), activate pool
- Inject pool questions into conversation
- During plan generation: Load pools via
feature.related_pool_ids
Pattern Application (Phase 6 - Specification):
- Load selected features
- Retrieve
related_patternsfor each feature - Filter patterns by cloud provider preference
- Apply pattern guidance to specification generation
- Include pattern-informed best practices in output
Phase Transitions
Transition Triggers
Phase transitions can be initiated in two ways:
1. API-Initiated Transitions
Endpoint: POST /phase/:projectId/transition
Process:
- Synchronous validation of transition eligibility (readiness threshold, required checks)
- Returns
202 Acceptedif validation passes - Asynchronous execution of exit actions → state change → entry actions
- Poll
GET /phase/:projectId/statusfor completion
Response Codes:
202 Accepted- Transition initiated, processing asynchronously400 Bad Request- Validation failed (readiness below threshold, missing requirements)409 Conflict- Concurrent transition detected (optimistic locking failure)500 Internal Server Error- Transition execution failed
2. Automatic Transitions
Trigger: PhaseTransitionDetector (in src/services/tasks/detectors/phase-transition-detector.ts)
Event Types:
question_completed- Discovery or planning question answeredtask_completed- User task or background task finishedbackground_task_completed- Research or processing task doneplan_generated- Question plan generation completefeature_curation_completed- Feature curation finishedresearch_completed- Research task tier completedesign_discovery_completed- Design question answering completeplanning_artifacts_generated- Blueprint and plan output readydiscovery_milestone_reached- Discovery phase progress threshold met
Detection Logic: On each event, the detector checks if the current phase readiness >= threshold AND all required checks pass. If true, automatically initiates transition via phase orchestrator.
Optimistic Locking
Phase transitions use optimistic locking to prevent concurrent modifications:
Mechanism: phase_version column with FOR UPDATE row lock during transition
Process:
-- Start transition
BEGIN;
SELECT phase, phase_version FROM projects WHERE id = ? FOR UPDATE;
-- Validate version matches expected
IF current_version != expected_version THEN
RAISE 409 Conflict
END IF
-- Execute transition
UPDATE projects SET phase = ?, phase_version = phase_version + 1
WHERE id = ? AND phase_version = ?;
COMMIT;Conflict Handling: Concurrent transitions raise 409 Conflict. Client must refetch project state and retry if appropriate.
Error Recovery
Repair Endpoint: POST /phase/:projectId/repair
Purpose: Re-run missing or failed entry actions for current phase without changing phase state
Use Cases:
- Entry action failed during transition
- Artifact generation incomplete
- Manual intervention required after phase change
Process: Idempotently executes entry actions for current phase, skipping actions whose artifacts already exist.
Phase Readiness Thresholds
Each phase has a specific readiness threshold that must be met before transition:
| Phase | Threshold | Required Checks | Score Formula |
|---|---|---|---|
| DISCOVERY | 50% | hasProjectName, hasDescription, hasPersona | max(0, round((1 - blockers/10) * 100)) from 7 core fields (name, description, goals, persona, domain, constraints, workflows) |
| FEATURE_SELECTION | 100% | featureCurationCompleted, featuresSelected | 0% → 40% (curation confirmed) → 70% (features selected) → 100% (plan generated) |
| PLAN_GENERATION | 70% | allTasksComplete, featureDrivenPlanGenerated, reviewPoliciesAssigned | 100% when plan exists + no pending tasks, else 0% |
| PLANNING | 100% | domainBlueprintGenerated, planOutputGenerated, noBlockingOpenQuestions | Blueprint exists (40%) + PlanOutput exists (40%) + No high-impact open questions (20%) |
| VISUAL_DISCOVERY | 80% | tier1ResearchMinimumMet, tier2OptionsAllComplete, designDiscoveryComplete | Tier1 research (40%, need 6/8) + Tier2 research (40%, need 6/6) + Design interactions (20%) |
| SPECIFICATION | 100% | allRequiredReviewsComplete, noBlockingItems | Required reviews (40%) + Ontology completeness (30%) + No blocking tasks/gaps (30%) |
Readiness Calculation: Implemented in PhaseOrchestrator.calculateReadiness() (lines 723-1341 in phase-orchestrator.ts). Each phase has a dedicated readiness method that queries relevant database state and computes a weighted score.
Threshold Enforcement: Transitions are blocked if readiness < threshold. API returns 400 with specific readiness details. Automatic detector skips transition attempt until threshold met.
Phase Entry Actions
Entry actions are idempotent operations executed when entering a new phase. They generate required artifacts and initialize phase-specific state.
FEATURE_SELECTION Entry Actions
-
Compile DiscoveryOutput - Extract structured data from discovery phase answers
- Queries:
graph_nodesWHEREproject_idANDnode_type = 'question' - Output:
projects.discovery_output(JSONB: archetype, domain, goals, users, constraints, workflows, criteria)
- Queries:
-
Generate FeatureCandidateSet - AI-curated feature suggestions based on discovery
- Input: DiscoveryOutput
- Output:
feature_curationstable with curated features - Agent: Uses specialized curation agent with feature ontology
-
Initialize FeatureSelectionState - Create selection tracking structure
- Output:
projects.feature_selection(JSONB: confirmed_features, prioritized_features, deferred_features)
- Output:
-
Trigger Background Feature Curation - Async refinement of feature set
- Background Task:
feature-curationworkflow - Updates: Feature scores, dependencies, implementation notes
- Background Task:
PLAN_GENERATION Entry Actions
-
Verify Feature Selection - Confirm at least one feature selected
- Check:
feature_selectionstable has entries - Fail: Cannot generate plan without features
- Check:
-
Generate Feature-Driven Question Plan - Create tailored question plan based on selected features
- Input: Selected features, discovery output, feature metadata
- Output:
question_plans+plan_sections+plan_questionstables - Logic: Feature requirements determine which question pools to include
-
Trigger onPlanGenerated Detection - Fire plan_generated event
- Event: Triggers automatic task generation for plan questions
- Detection: Creates user tasks for each plan question requiring input
PLANNING Entry Actions
-
Generate DomainBlueprint - High-level architecture and domain model
- Input: Discovery output, selected features, answered plan questions
- Output:
projects.domain_blueprint(JSONB: entities, relationships, bounded contexts, services) - Agent: Architecture synthesis agent
-
Generate PlanOutput - Implementation execution plan
- Input: DomainBlueprint, feature details, technical requirements
- Output:
projects.plan_output(JSONB: phases, milestones, dependencies, resource estimates) - Agent: Planning agent with project management expertise
-
Trigger onPlanningArtifactsGenerated Detection - Fire planning_artifacts_generated event
- Event: Signals completion of planning phase artifacts
- Detection: Checks readiness for Visual Discovery transition
VISUAL_DISCOVERY Entry Actions
-
Generate BlockRequirements - Identify needed UI components
- Input: Selected features, planning artifacts, domain entities
- Output:
projects.block_requirements(JSONB: required_blocks, complexity_preferences, style_preferences) - Logic: Maps features to design surfaces, surfaces to blocks
-
Generate DesignQuestionPlan - Create design-specific question set
- Input: BlockRequirements, feature UX needs
- Output:
projects.design_question_plan(JSONB: sections with block-based questions) - Sections: Theme/Style, Auth Pages, Marketing, Navigation, Dashboard, Dialogs, E-commerce
-
Trigger Tier 1 Research Tasks - Start background visual research
- Tasks: market-research, design-intelligence, technical-feasibility, competitive-analysis, visual-curation, architecture-analysis, business-strategy, product-management
- Parallel Execution: All 8 Tier 1 tasks run concurrently
- Completion: 6 of 8 required for Tier 1 completion, triggers Tier 2
SPECIFICATION Entry Actions
- Trigger User Story Task Chain - Generate implementation-ready stories (non-blocking)
- First Task:
backlog_draft(generates epics and stories) - Chain: backlog_draft → backlog_gap_scan || story_surface_mapper → surface_block_mapper → spec_coverage_validator
- Non-Blocking: Export can proceed even if stories not fully generated (warning issued)
- First Task:
Entry Action Idempotency: All entry actions check for existing artifacts before generation:
// Example from FEATURE_SELECTION entry
const existingCuration = await sql`SELECT id FROM feature_curations WHERE project_id = ?`
if (existingCuration) {
logger.info('Feature curation already exists, skipping')
return
}
// Proceed with generation...Failure Handling: If an entry action fails, the transition is rolled back:
- Phase remains in previous state
- Error tracked in
phase_transition_errorstable phase_versionNOT incremented (allows retry)- User notified via API response or event
Phase Inputs & Outputs
Each phase consumes artifacts from previous phases and produces new artifacts for subsequent phases:
| Phase | Input Artifacts | Output Artifacts | Storage |
|---|---|---|---|
| DISCOVERY | User answers to discovery questions | DiscoveryOutput - Structured representation of project intent (archetype, domain, goals, users, constraints, workflows, success criteria) | projects.discovery_output (JSONB) |
| FEATURE_SELECTION | DiscoveryOutput from discovery phase | FeatureCandidateSet - AI-curated feature suggestionsFeatureSelectionState - User's selected/deferred features | feature_curations (curated features)projects.feature_selection (JSONB) |
| PLAN_GENERATION | FeatureSelectionState from feature selection | QuestionPlan - Tailored question set based on selected features (sections, questions, readiness tracking) | question_plans, plan_sections, plan_questions tables |
| PLANNING | Answered plan_questions from plan generation | DomainBlueprint - High-level architecture and domain modelPlanOutput - Implementation execution plan with phases and milestones | projects.domain_blueprint (JSONB)projects.plan_output (JSONB) |
| VISUAL_DISCOVERY | Planning data (blueprint, plan output, features) | DesignQuestionPlan - Design-specific questions with block optionsResearchOutputs - 8 Tier-1 research outputs, 6 Tier-2 generation outputs | projects.block_requirements (JSONB)projects.design_question_plan (JSONB)background_tasks (research results) |
| SPECIFICATION | All prior artifacts (discovery through visual discovery) | UserStories - Implementation-ready user stories with traceabilityExportBundle - Complete specification in multiple formats (JSON, Markdown, YAML) | user_story_backlogs, user_story_epics, user_stories tablesexport_packs, project_specs tables |
Artifact Flow: Each phase's output artifacts become input artifacts for the next phase. This creates a progressive refinement pipeline where early-phase decisions (like domain selection in Discovery) influence later-phase outputs (like design component selection in Visual Discovery).
Artifact Validation: Phase entry actions validate that required input artifacts exist before proceeding. Missing artifacts cause transition failures, preventing progression with incomplete data.
Key Services
Flow Services (43 files in /src/services/flow/)
| Service | Purpose |
|---|---|
question-flow-orchestrator.ts | Main Q&A flow coordinator |
variant-enhancer.ts | AI enhancement of question variants |
ontology-tracker.ts | Real-time ontology entity tracking |
suggestion-generator.ts | Post-answer suggestions |
completion-manager.ts | Question completion tracking |
template-instantiator.ts | Template → graph_nodes |
answer-prefill-generator.ts | Prefill candidate generation |
persona-matcher.ts | Persona-based auto-answering |
technical-decision-engine.ts | Tech decision automation |
Task Services (13 files)
| Service | Purpose |
|---|---|
user-task-service.ts | User task CRUD |
task-detector-service.ts | Automatic task detection |
question-task-generator.ts | Tasks from questions |
output-binding-service.ts | Post-action data binding |
Background Services
| Service | Purpose |
|---|---|
worker.ts | Background task processor (10s poll, max 2 concurrent) |
post-answer-processor.ts | Async post-answer pipeline |
feature-curation-processor.ts | Feature curation jobs |
Permissions Services (8 files in /src/services/permissions/)
| Service | Purpose |
|---|---|
role-manager.ts | RBAC role management and capability tracking |
policy-engine.ts | ABAC policy evaluation with condition parsing |
permission-validator.ts | Real-time permission enforcement |
Template Services (12 files in /src/services/templates/)
| Service | Purpose |
|---|---|
variable-extractor.ts | Automatic variable extraction from templates |
mapping-resolver.ts | Multi-source variable mapping resolution |
template-renderer.ts | Multi-format rendering (PDF, HTML, DOCX, Email) |
Workflow Services (5 files in /src/services/workflows/)
| Service | Purpose |
|---|---|
bpmn-parser.ts | BPMN 2.0 XML parsing and storage |
bpmn-validator.ts | Multi-layer validation (structural, semantic, platform) |
workflow-normalizer.ts | Normalized JSON workflow model generation |
Integration Services (8 files in /src/services/integrations/)
| Service | Purpose |
|---|---|
curated-integrations.ts | Stripe, Gmail, SendGrid, Auth0, S3, etc. |
contract-importer.ts | OpenAPI/AsyncAPI import and parsing |
mapping-service.ts | Schema mapping with transform expressions |
Gaps Services (6 files in /src/services/gaps/)
| Service | Purpose |
|---|---|
detector-runner.ts | Gaps detection orchestration |
detector-registry.ts | Detector registration and lifecycle |
task-synthesizer.ts | Deterministic task generation from findings |
Export Services (4 files in /src/services/export/)
| Service | Purpose |
|---|---|
spec-compiler.ts | Multi-phase export bundle compilation |
direct-export-generator.ts | Bundle packaging with checksums |
stream-events.ts | Real-time export progress streaming |
Critical Files
The Most Important File
src/services/flow/phase-orchestrator.ts (1,730 lines)
This file is the heart of the system. It handles:
- All phase lifecycle management
- Transition validation and execution
- Readiness calculation per phase
- Entry action orchestration
- State machine logic
Note: The Mastra Workflow Refactor (in progress) aims to decompose this monolith into testable workflow steps.
Other Critical Files
| File | Purpose |
|---|---|
src/api/routes/flow.ts | Flow orchestration endpoints |
src/api/routes/projects.ts | Project CRUD operations |
src/core/agents/orchestration/ | Agent coordination logic |
src/workflows/definitions/ | Mastra workflow definitions |
src/services/spec-compiler/ | SpecV1 compilation |
mastra.config.ts | Agent and workflow registration |
src/services/permissions/policy-engine.ts | ABAC policy evaluation |
src/services/templates/template-renderer.ts | Multi-format template rendering |
src/services/workflows/bpmn-validator.ts | Workflow validation engine |
src/services/integrations/contract-importer.ts | OpenAPI/AsyncAPI import |
src/services/gaps/detector-runner.ts | Gaps detection orchestration |
src/services/export/spec-compiler.ts | Export bundle compilation |
Graph-Engine Service
A separate service running on port 4002 with its own dedicated PostgreSQL database. Added in the automation-consolidation branch.
Role: Maintains the Unified Project Graph — a bidirectional spec↔code knowledge graph that drives code generation order, convergence checks, and pattern management.
Next.js (4001) ──► Backend/API (4000)
│
▼
Graph-Engine (4002) ◄── Codegen workers
(spec↔code graph,
pattern system,
convergence checks)
│
▼
Graph-Engine DB
(context_artifacts,
context_artifact_dependencies)API Groups
| Route Prefix | Purpose |
|---|---|
/api/graph | Traversal, coverage, drift, blast radius queries |
/api/ingest | Spec ingestion |
/api/impl | Impl node planning |
/api/generation | Batch code generation runs |
/api/convergence | Coverage, drift, and security checks |
/api/pipeline | Pipeline debugger traces |
/api/flywheel | Pattern promotion and analytics |
/api/patterns | Pattern CRUD and matching |
See Unified Project Graph for the full graph reference.
Last Updated: March 2026