Kit System
The Kit System is Praetor's extensible code generation and platform integration framework. Kits are self-contained packages that generate platform-specific configuration, code, and documentation based
Kit System
Notice: This section describes the legacy Kit System architecture. The current Kit protocol — using the CHECK/SPEC/PLAN/APPLY/TEST/VERIFY/REGISTER lifecycle — is documented in the Kit Execution guide. The historical content below remains for reference.
Table of Contents
- Overview
- Architecture
- Kit Package Format
- Operations Lifecycle
- Platform Integration
- Code Generation
- Testing Framework
- API Reference
- Kit Development
- Best Practices
Overview
The Kit System is Praetor's extensible code generation and platform integration framework. Kits are self-contained packages that generate platform-specific configuration, code, and documentation based on business requirements.
What is a Kit?
A Kit is a reusable package containing:
- Templates - Liquid templates for generating outputs
- Knowledge - Domain-specific documentation and constraints
- Validation - JSON schemas and custom validation logic
- Discovery - Interactive Q&A pools for gathering requirements
- Operations - Seven distinct operations for complete lifecycle management
Kit Package Structure
├── kit.yaml # Manifest with metadata and configuration
├── prompts/ # Liquid templates for code generation
│ ├── config.liquid
│ ├── setup-guide.liquid
│ └── api-spec.liquid
├── knowledge/ # Domain knowledge for AI context
│ └── domain/
│ ├── index.yaml
│ ├── platform.yaml
│ ├── constraints.yaml
│ └── glossary.yaml
├── validation/ # JSON schemas and validation rules
│ ├── config.schema.json
│ └── custom.yaml
├── discovery/ # Interactive Q&A definitions
│ ├── pool-1.yaml
│ └── pool-2.yaml
└── tests/ # Test fixtures for validation
├── fixture-basic.yaml
└── fixture-advanced.yamlUse Cases
1. Platform Configuration Generate platform-specific setup (HubSpot CRM, Salesforce, Stripe)
2. Framework Setup Generate project scaffolding (Next.js, Express, Django)
3. Documentation Generation Create standards-compliant documentation (CLAUDE.md, API docs, guides)
4. Integration Configuration Build API client code and authentication flows
5. Infrastructure as Code Generate Terraform, Kubernetes, or Docker configurations
Architecture
System Layers
┌─────────────────────────────────────────────────────────────────┐
│ FRONTEND LAYER │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Kit Catalog │ │ Kit Execution│ │ Credential Management│ │
│ │ Browser │ │ UI │ │ Interface │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ API LAYER │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ /api/kits │ │/api/kits/:id │ │ /api/kits/credentials│ │
│ │ │ │ /execute │ │ │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ KIT EXECUTOR │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Seven Operations Lifecycle: │ │
│ │ 1. SPEC → Identify requirements │ │
│ │ 2. CHECK → Verify platform connectivity │ │
│ │ 3. DISCOVER→ Gather existing resources │ │
│ │ 4. PLAN → Generate configuration/code │ │
│ │ 5. VALIDATE→ Verify outputs against schemas │ │
│ │ 6. APPLY → Deploy to platform (optional) │ │
│ │ 7. VERIFY → Confirm successful deployment │ │
│ └──────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────┐
│ CORE SERVICES │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Kit Loader │ │ Kit Registry │ │ Kit Factory (AI) │ │
│ │ │ │ │ │ │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
│ │ Runtime │ │ Recovery │ │ Connectors │ │
│ │ Services │ │ Services │ │ (Platform APIs) │ │
│ └──────────────┘ └──────────────┘ └──────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘Key Components
1. Kit Loader
Location: src/kits/kit-loader.ts
Loads and validates Kit packages from the filesystem or database:
- Parses
kit.yamlmanifest - Validates against Kit schema
- Loads knowledge index
- Verifies referenced files exist
2. Kit Executor
Location: src/kits/kit-executor.ts
Orchestrates the seven-operation lifecycle:
- Manages execution state
- Handles error recovery
- Tracks progress
- Returns structured results
3. Operation Handlers
Location: src/kits/operations/*.ts
Seven specialized handlers:
spec.ts- Requirements analysischeck.ts- Connectivity verificationdiscover.ts- Resource discoveryplan.ts- Code generationvalidate.ts- Output validationapply.ts- Platform deploymentverify.ts- Deployment verification
4. Platform Connectors
Location: src/kits/connectors/*.ts
Platform API integration:
platform-connector.ts- Base connector interfaceactivepieces-adapter.ts- ActivePieces integrationapi-client.ts- HTTP client with paginationrate-limiter.ts- Token bucket rate limitingcredential-manager.ts- Secure credential storage
5. Runtime Services
Location: src/kits/runtime/*.ts
Execution utilities:
liquid-renderer.ts- Template renderingschema-validator.ts- JSON schema validationjsonata-transformer.ts- Data transformationconstraint-validator.ts- Custom validation rulesknowledge-loader.ts- Domain knowledge assembly
6. Recovery Services
Location: src/kits/recovery/*.ts
Resilience and error handling:
deterministic-retry.ts- Exponential backoff retrygraceful-degradation.ts- Fallback strategiescognitive-recovery.ts- AI-powered error recoveryerror-classifier.ts- Error categorization
Kit Package Format
Manifest (kit.yaml)
name: hubspot-crm
displayName: HubSpot CRM Setup Kit
description: Configure HubSpot CRM with custom properties and pipelines
version: 1.0.0
type: platform # platform | file | hybrid | api
icon: hubspot
author: praetor
status: active # active | beta | deprecated
# Platform metadata (for platform/api kits)
platform:
name: HubSpot
category: CRM
website: https://www.hubspot.com
editions:
- id: starter
label: HubSpot Starter
monthlyPrice: 20
features: [contacts, deals, custom_properties]
limits:
custom_properties: 1000
pipelines: 2
# Supported operations
supportedOperations:
- spec
- check
- discover
- plan
- validate
- apply
- verify
# Input requirements
requires:
inputs:
compiledSpec: optional
businessProfile: required
credentials:
type: api_key
fields:
- name: api_key
type: password
required: true
description: HubSpot Private App Access Token
profileSections:
- section: company
required: true
reason: Company name for CRM naming conventions
minimumObligations:
- pipeline-structure-confirmed
- property-naming-confirmed
# Output definitions
outputs:
- id: pipeline-config
filename: pipeline-configuration.json
type: data
format: json
description: Deal pipeline stages and properties
template: prompts/pipeline-config.liquid
schema: validation/pipeline.schema.json
knowledgeFiles:
- features.yaml
- id: setup-guide
filename: SETUP_GUIDE.md
type: document
format: markdown
description: Step-by-step setup instructions
template: prompts/setup-guide.liquid
dependsOn:
- pipeline-config
# Discovery pools
discoveryPools:
- hubspot-pipeline-mapping
- hubspot-property-definitions
# Execution configuration
execution:
plan:
source: prompts
method: ai-generate
validate:
checks:
- schema
- constraints
apply:
method: step-by-step
transactional: false
respectRateLimits: true
retryPolicy:
retryable: [429, 500, 502, 503]
maxRetries: 3
backoffMs: [1000, 5000, 15000]Template System
Kits use Liquid templating for code generation:
{%- comment -%}
prompts/pipeline-config.liquid
Generates HubSpot deal pipeline configuration
{%- endcomment -%}
# HubSpot Deal Pipeline Configuration
## Context
**Company:** {{ businessProfile.company.name }}
**Industry:** {{ businessProfile.company.industry }}
**Sales Cycle:** {{ businessProfile.sales.cycle }}
## Task
Generate a JSON configuration for HubSpot deal pipelines that includes:
1. Pipeline name based on company and sales process
2. Deal stages matching the sales cycle
3. Stage-specific properties
4. Win probability per stage
## Constraints
- Must match schema: {{ schema.pipeline }}
- Stage names: short, clear, action-oriented
- Display order: 1-indexed, sequential
- Win probability: 0-100, increasing with stage
## Output Format
```json
{
"name": "string",
"stages": [
{
"label": "string",
"displayOrder": number,
"probability": number
}
]
}Business Context
{{ knowledgeContext.platform }}
{{ knowledgeContext.constraints }}
Generate the configuration now.
### Knowledge System
Knowledge files provide domain context to AI models:
**knowledge/domain/index.yaml**
```yaml
always:
- platform.yaml
- constraints.yaml
- glossary.yaml
perOutput:
pipeline-config:
files:
- features.yaml
- data-model.yaml
property-definitions:
files:
- data-model.yamlknowledge/domain/platform.yaml
# HubSpot Platform Overview
platform: HubSpot CRM
version: 2024
capabilities:
- Contact management
- Deal pipelines (up to 15)
- Custom properties (up to 10,000)
- Workflows and automation
- Email integration
api:
version: v3
base_url: https://api.hubapi.com
authentication: API key (Private App)
rate_limits:
per_second: 10
per_10_seconds: 100
burst: 150Validation System
JSON Schema Validation
// validation/pipeline.schema.json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": ["name", "stages"],
"properties": {
"name": {
"type": "string",
"minLength": 3,
"maxLength": 100
},
"stages": {
"type": "array",
"minItems": 2,
"maxItems": 15,
"items": {
"type": "object",
"required": ["label", "displayOrder"],
"properties": {
"label": { "type": "string" },
"displayOrder": { "type": "integer", "minimum": 1 },
"probability": { "type": "integer", "minimum": 0, "maximum": 100 }
}
}
}
}
}Custom Validation (YAML)
# validation/custom.yaml
constraints:
- id: unique-stage-names
type: uniqueness
field: stages[].label
message: Stage names must be unique
- id: sequential-display-order
type: sequence
field: stages[].displayOrder
start: 1
message: Display order must be sequential starting from 1
- id: increasing-probability
type: monotonic
field: stages[].probability
direction: increasing
message: Win probability should increase with each stageDiscovery Pools
Interactive Q&A for gathering requirements:
# discovery/pipeline-mapping.yaml
id: hubspot-pipeline-mapping
name: Pipeline Structure Mapping
description: Understand your sales pipeline structure
weight: 100
questions:
- id: pipeline-structure-confirmed
type: confirmation
prompt: |
Based on your business profile, I recommend this pipeline structure:
{{ recommendedPipeline | yaml }}
Does this match your sales process?
prefill:
source: businessProfile.sales.process
rule: extract-stages
onConfirm:
obligation: pipeline-structure-confirmed
onReject:
followUp: custom-pipeline-structure
- id: custom-pipeline-structure
type: structured-list
prompt: Please describe your pipeline stages in order:
itemStructure:
- field: stage_name
label: Stage Name
type: text
- field: typical_duration
label: Typical Duration (days)
type: numberOperations Lifecycle
1. SPEC Operation
Purpose: Analyze kit requirements and validate inputs
// Returns KitSpec object
interface KitSpec {
name: string;
version: string;
type: 'platform' | 'file' | 'hybrid' | 'api';
supportedOperations: Operation[];
inputs: {
compiledSpec?: CredentialSpec;
businessProfile?: ProfileRequirement;
credentials?: CredentialSpec;
};
profileRequirements: ProfileRequirement[];
outputs: OutputRequirement[];
discoveryPools: string[];
obligations: string[];
}Usage:
import { loadKit } from './kits/kit-loader';
import { executeSpec } from './kits/operations/spec';
const kit = await loadKit('/path/to/hubspot-crm');
const spec = executeSpec(kit);
console.log(spec.inputs.credentials); // API key required
console.log(spec.profileRequirements); // [company section]2. CHECK Operation
Purpose: Verify platform connectivity and credentials
interface CheckResult {
connected: boolean;
platformEdition?: string;
capabilities: string[];
credentialValid: boolean;
metadata?: Record<string, unknown>;
}Usage:
import { executeCheck } from './kits/operations/check';
const checkResult = await executeCheck(kit, {
credential: storedCredential,
connector: platformConnector,
});
if (!checkResult.connected) {
throw new Error('Cannot connect to HubSpot API');
}3. DISCOVER Operation
Purpose: Gather existing platform resources
interface DiscoverResult {
resources: DiscoveredResource[];
summary: string;
metadata?: Record<string, unknown>;
}
interface DiscoveredResource {
id: string;
type: string;
name: string;
metadata: Record<string, unknown>;
}Usage:
import { executeDiscover } from './kits/operations/discover';
const discoverResult = await executeDiscover(kit, {
credential: storedCredential,
connector: platformConnector,
scope: ['pipelines', 'properties'],
});
console.log(`Found ${discoverResult.resources.length} existing resources`);4. PLAN Operation
Purpose: Generate code/configuration outputs
interface PlanResult {
outputs: GeneratedOutput[];
totalTokenUsage: TokenUsage;
durationMs: number;
errors: string[];
}
interface GeneratedOutput {
outputId: string;
filename: string;
content: string;
tokenUsage: TokenUsage;
}Usage:
import { executePlan } from './kits/operations/plan';
const { outputs } = await executePlan(kit, {
knowledge: assembledKnowledge,
businessProfile: userProfile,
discoveryAnswers: confirmedAnswers,
}, {
generate: aiGenerateFunction,
});
// outputs[0].content = JSON configuration
// outputs[1].content = Markdown documentation5. VALIDATE Operation
Purpose: Verify outputs against schemas and constraints
interface KitValidateResult {
valid: boolean;
errors: ValidationError[];
warnings: ValidationWarning[];
outputResults: Record<string, OutputValidationResult>;
summary: {
totalChecks: number;
totalErrors: number;
totalWarnings: number;
};
}Usage:
import { executeValidate } from './kits/operations/validate';
const validationResult = await executeValidate(kit, outputs);
if (!validationResult.valid) {
console.error('Validation failed:', validationResult.errors);
// Optionally trigger auto-repair
}6. APPLY Operation
Purpose: Deploy configuration to platform
interface ApplyResult {
applied: boolean;
steps: ApplyStep[];
rollback?: RollbackPlan;
}
interface ApplyStep {
id: string;
action: 'create' | 'update' | 'delete';
resource: string;
status: 'success' | 'failed' | 'skipped';
response?: unknown;
error?: string;
}Usage:
import { executeApply } from './kits/operations/apply';
const applyResult = await executeApply(kit, {
outputs: validatedOutputs,
credential: storedCredential,
connector: platformConnector,
options: {
dryRun: false,
createCheckpoint: true,
},
});7. VERIFY Operation
Purpose: Confirm successful deployment
interface VerifyResult {
verified: boolean;
checks: VerifyCheck[];
}
interface VerifyCheck {
id: string;
name: string;
status: 'pass' | 'fail';
message?: string;
}Usage:
import { executeVerify } from './kits/operations/verify';
const verifyResult = await executeVerify(kit, {
applyResult,
credential: storedCredential,
connector: platformConnector,
});
if (verifyResult.verified) {
console.log('✅ Deployment successful');
}Platform Integration
Connector Interface
All platform integrations implement PlatformConnector:
interface PlatformConnector {
// Connectivity
testConnection(credential: StoredCredential): Promise<ConnectionTestResult>;
// Discovery
discoverResources(options: DiscoverOptions): Promise<DiscoveredResource[]>;
// CRUD operations
createResource(type: string, data: unknown): Promise<ResourceResponse>;
updateResource(type: string, id: string, data: unknown): Promise<ResourceResponse>;
deleteResource(type: string, id: string): Promise<void>;
getResource(type: string, id: string): Promise<unknown>;
// Bulk operations
batchCreate(resources: BatchResource[]): Promise<BatchResult>;
batchUpdate(resources: BatchResource[]): Promise<BatchResult>;
}ActivePieces Integration
Kits can leverage ActivePieces connectors for 200+ platforms:
import { ActivePiecesAdapter } from './connectors/activepieces-adapter';
const adapter = new ActivePiecesAdapter({
apiUrl: process.env.ACTIVEPIECES_API_URL,
apiKey: process.env.ACTIVEPIECES_API_KEY,
});
// Use ActivePieces pieces as platform connectors
const hubspotConnector = await adapter.createConnector('hubspot', {
credential: storedCredential,
});
const contacts = await hubspotConnector.listContacts({
limit: 100,
});Rate Limiting
Built-in token bucket rate limiter:
import { RateLimiter } from './connectors/rate-limiter';
const limiter = new RateLimiter({
perSecond: 10,
per10Seconds: 100,
queryLimit: 10000,
burstCapacity: 150,
});
// Automatically throttles requests
await limiter.waitForCapacity(1, 50); // weight=1, queryCost=50
await apiClient.makeRequest();Credential Management
Secure storage with encryption:
import { storeCredential, getCredential } from './connectors/credential-manager';
// Store encrypted credential
const credentialId = await storeCredential({
tenantId,
projectId,
kitId: 'hubspot-crm',
credentialType: 'api_key',
data: {
api_key: userProvidedKey,
},
});
// Retrieve and decrypt
const credential = await getCredential(credentialId);Code Generation
AI-Powered Generation
The PLAN operation uses AI models to generate outputs:
// AI generation function signature
type AiGenerateFunction = (
prompt: string,
options?: { maxTokens?: number }
) => Promise<{
content: string;
tokenUsage: TokenUsage;
}>;
// Example with Gemini 2.0
import { google } from '@ai-sdk/google';
import { generateText } from 'ai';
const generate: AiGenerateFunction = async (prompt, options) => {
const result = await generateText({
model: google('gemini-2.0-flash'),
prompt,
maxTokens: options?.maxTokens ?? 4000,
});
return {
content: result.text,
tokenUsage: {
prompt: result.usage.promptTokens,
completion: result.usage.completionTokens,
total: result.usage.totalTokens,
},
};
};Context Assembly
Knowledge files are assembled into prompts:
import { assembleContext } from './runtime/context-assembler';
const context = await assembleContext(kit, {
outputId: 'pipeline-config',
businessProfile: userProfile,
discoveryAnswers: answers,
previousOutputs: [], // For dependent outputs
});
// Context includes:
// - Template content
// - Knowledge files
// - Business profile
// - Discovery answers
// - Previous outputs (for dependencies)Template Rendering
Post-generation, templates can reference generated data:
import { renderTemplate } from './runtime/liquid-renderer';
const content = await renderTemplate(templateContent, {
businessProfile: userProfile,
generatedConfig: JSON.parse(outputs[0].content),
timestamp: new Date().toISOString(),
});Output Dependencies
Outputs can depend on previously generated outputs:
outputs:
- id: api-client
template: prompts/client.liquid
- id: api-tests
template: prompts/tests.liquid
dependsOn: [api-client] # Has access to api-client content
- id: readme
template: prompts/readme.liquid
dependsOn: [api-client, api-tests] # Has access to bothThe executor generates outputs in dependency order and injects previous outputs into subsequent prompts.
Testing Framework
Three-Tier Testing
Tier 1: Protocol Acceptance Tests
- Structural validation
- No credentials needed
- Runs in
<5seconds - ~30 checks per kit
import { runProtocolAcceptance } from './testing/protocol-acceptance';
const result = await runProtocolAcceptance('/path/to/kit');
console.log(`${result.passCount} passed, ${result.failCount} failed`);
// Checks: manifest valid, templates exist, schemas valid, etc.Tier 2: Fixture Tests
- Offline AI tests with mock responses
- Tests SPEC → PLAN → VALIDATE pipeline
- Duration: 10-60s per fixture
- Validates business logic
import { runFixtureTests } from './testing/fixture-runner';
const result = await runFixtureTests('/path/to/kit', {
generate: mockAiFunction,
filterName: 'saas-company',
});
console.log(`${result.passed}/${result.totalFixtures} fixtures passed`);Tier 3: Integration Tests
- Full pipeline against real platform APIs
- Requires valid credentials
- Sandbox environment
- Duration: 1-10 minutes
- Includes cleanup
import { runIntegrationTest } from './testing/integration-runner';
const result = await runIntegrationTest({
packagePath: '/path/to/kit',
credential: sandboxCredential,
businessProfile: testProfile,
cleanup: true,
timeoutMs: 300000,
});
console.log(`Status: ${result.status}`);Test Fixtures
Define test scenarios in YAML:
# tests/fixture-saas-company.yaml
name: SaaS Company - Standard Pipeline
description: Test kit with typical SaaS sales process
businessProfile:
company:
name: Acme SaaS Inc
industry: saas
size: 50-100
sales:
cycle: medium
process: lead → qualified → demo → proposal → closed
expectedOutputs:
- pipeline-config
- setup-guide
assertions:
- outputId: pipeline-config
type: contains
value: "Acme SaaS"
- outputId: pipeline-config
type: min-length
value: 500
- outputId: setup-guide
type: contains
value: "Step 1"API Reference
Kit Management Endpoints
List Kits
GET /api/kitsReturns all available kits with metadata.
Response:
{
"kits": [
{
"id": "hubspot-crm",
"name": "HubSpot CRM Setup Kit",
"type": "platform",
"status": "active",
"icon": "hubspot",
"supportedOperations": ["spec", "check", "discover", "plan", "validate", "apply", "verify"]
}
]
}Get Kit Details
GET /api/kits/:kitIdReturns full kit manifest and specifications.
Execute Kit
POST /api/kits/:kitId/executeExecutes kit operations.
Request:
{
"operations": ["spec", "check", "plan", "validate"],
"inputs": {
"businessProfile": { ... },
"discoveryAnswers": { ... },
"credentialId": "cred_123"
},
"options": {
"dryRun": false,
"maxRepairAttempts": 3
}
}Response:
{
"executionId": "exec_abc123",
"status": "completed",
"outputs": [
{
"outputId": "pipeline-config",
"filename": "pipeline-configuration.json",
"content": "{...}",
"tokenUsage": { "total": 550 }
}
],
"spec": { ... },
"checkResult": { "connected": true },
"planResult": { ... },
"validationResult": { "valid": true }
}Credential Management
Store Credential
POST /api/kits/credentialsRequest:
{
"kitId": "hubspot-crm",
"credentialType": "api_key",
"data": {
"api_key": "secret_key_here"
}
}List Credentials
GET /api/kits/credentials?kitId=hubspot-crmDelete Credential
DELETE /api/kits/credentials/:credentialIdKit Factory (AI Generation)
Generate Kit
POST /api/kits/factoryAI-powered kit generation from platform documentation.
Request:
{
"platformName": "Stripe",
"platformUrl": "https://stripe.com",
"openApiSpecUrl": "https://api.stripe.com/openapi.yaml",
"kitType": "platform",
"category": "payments"
}Returns task ID for async generation. Kit is created at src/kits/packages/{platformId}/.
Kit Development
Creating a New Kit
1. Generate Kit Structure
mkdir -p src/kits/packages/my-kit/{prompts,knowledge/domain,validation,discovery,tests}2. Create Manifest
# kit.yaml
name: my-kit
displayName: My Kit
description: Brief description
version: 1.0.0
type: file
author: your-name
status: active
supportedOperations:
- spec
- plan
- validate
requires:
inputs:
businessProfile: required
outputs:
- id: config
filename: config.json
type: data
format: json
template: prompts/config.liquid3. Create Templates
{%- comment -%}
prompts/config.liquid
{%- endcomment -%}
Generate a JSON configuration for {{ businessProfile.company.name }}.
Requirements:
- Include company name
- Follow JSON format
- Be concise
Output the configuration now.4. Add Knowledge (Optional)
# knowledge/domain/index.yaml
always:
- constraints.yaml# knowledge/domain/constraints.yaml
constraints:
- Keep configurations minimal
- Use clear naming
- Follow JSON conventions5. Add Validation (Optional)
// validation/config.schema.json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": ["name"],
"properties": {
"name": { "type": "string" }
}
}6. Create Test Fixtures
# tests/fixture-basic.yaml
name: basic-test
description: Test basic functionality
businessProfile:
company:
name: Test Company
expectedOutputs:
- config
assertions:
- outputId: config
type: contains
value: "Test Company"7. Test Your Kit
# Protocol acceptance
npm run test:kit:protocol -- src/kits/packages/my-kit
# Fixture tests
npm run test:kit:fixtures -- src/kits/packages/my-kit
# Integration test (if applicable)
npm run test:kit:integration -- src/kits/packages/my-kitUsing Kit Factory
Generate kits automatically from platform documentation:
import { generateKitPackage } from './kits/kit-factory';
const result = await generateKitPackage({
platformId: 'stripe',
platformName: 'Stripe',
category: 'payments',
website: 'https://stripe.com',
kitType: 'platform',
openApiSpec: 'https://api.stripe.com/openapi.yaml',
rawDocs: `Stripe API documentation content...`,
outputDir: './src/kits/packages/stripe',
generate: aiGenerateFunction,
});
console.log(`Generated kit: ${result.kitId}`);
console.log(`Quality: ${result.qualityTier}`);
console.log(`Files created: ${result.filesGenerated.length}`);Best Practices
Kit Design
-
Single Responsibility
- One kit per platform or framework
- Clear, focused purpose
- Avoid combining unrelated functionality
-
Comprehensive Knowledge
- Include platform constraints
- Document naming conventions
- Provide examples in knowledge files
-
Robust Validation
- Use JSON schemas for structure
- Add custom constraints for business logic
- Include clear error messages
-
Granular Outputs
- Break complex outputs into smaller files
- Use dependencies for ordering
- Enable selective generation with
outputFilter
-
Clear Documentation
- Document each output's purpose
- Provide setup instructions
- Include troubleshooting tips
Template Design
-
Provide Context
- Start with business context
- Reference knowledge files
- Explain the goal clearly
-
Define Constraints
- Be specific about requirements
- Reference schemas
- List what to avoid
-
Use Examples
- Show expected output format
- Provide sample data
- Demonstrate edge cases
-
Leverage Previous Outputs
- Reference dependent outputs in prompt
- Ensure consistency across files
- Build on established patterns
Error Handling
-
Graceful Degradation
- Provide fallback behaviors
- Continue with partial results when possible
- Log warnings instead of failing
-
Cognitive Recovery
- Use AI to interpret and fix errors
- Attempt repairs automatically
- Limit repair attempts (max 3)
-
Clear Error Messages
- Include actionable guidance
- Reference documentation
- Suggest next steps
Security
-
Credential Management
- Always encrypt credentials
- Use environment variables
- Never log credentials
- Rotate keys regularly
-
Validation
- Validate all inputs
- Sanitize output before deployment
- Check for injection attacks
-
Rate Limiting
- Respect platform limits
- Implement backoff strategies
- Monitor usage
-
Dry Run First
- Test with
dryRun: true - Review changes before applying
- Implement rollback plans
- Test with
Performance
-
Optimize Prompts
- Keep templates concise
- Avoid redundant context
- Use appropriate token limits
-
Parallel Execution
- Generate independent outputs in parallel
- Use batch operations
- Cache responses when possible
-
Incremental Updates
- Detect unchanged resources
- Skip unnecessary operations
- Use checksums for validation
Appendix
File Locations
Core Services:
- Kit Loader:
src/kits/kit-loader.ts - Kit Executor:
src/kits/kit-executor.ts - Kit Registry:
src/kits/kit-registry.ts - Kit Factory:
src/kits/kit-factory.ts
Operations:
src/kits/operations/spec.tssrc/kits/operations/check.tssrc/kits/operations/discover.tssrc/kits/operations/plan.tssrc/kits/operations/validate.tssrc/kits/operations/apply.tssrc/kits/operations/verify.ts
Connectors:
src/kits/connectors/platform-connector.tssrc/kits/connectors/activepieces-adapter.tssrc/kits/connectors/api-client.tssrc/kits/connectors/rate-limiter.tssrc/kits/connectors/credential-manager.ts
Runtime:
src/kits/runtime/liquid-renderer.tssrc/kits/runtime/schema-validator.tssrc/kits/runtime/jsonata-transformer.tssrc/kits/runtime/constraint-validator.tssrc/kits/runtime/knowledge-loader.tssrc/kits/runtime/context-assembler.ts
Recovery:
src/kits/recovery/deterministic-retry.tssrc/kits/recovery/graceful-degradation.tssrc/kits/recovery/cognitive-recovery.tssrc/kits/recovery/error-classifier.ts
Discovery:
src/kits/discovery/pool-activator.tssrc/kits/discovery/obligation-tracker.tssrc/kits/discovery/prefill-engine.ts
Testing:
src/kits/testing/protocol-acceptance.tssrc/kits/testing/fixture-runner.tssrc/kits/testing/integration-runner.ts
Database:
- Migrations:
src/db/migrations/208-212_*.sql - Tables:
kit_definitions,kit_credentials,kit_executions,kit_execution_outputs,kit_discover_cache
API Routes:
src/api/routes/kits.ts
Frontend:
testing-ui/src/app/kits/page.tsxtesting-ui/src/app/kits/[kitId]/page.tsxtesting-ui/src/app/kits/credentials/page.tsxtesting-ui/src/app/kits/executions/[executionId]/page.tsxtesting-ui/src/components/kits/KitCard.tsxtesting-ui/src/hooks/useKitExecution.ts
Configuration
Environment Variables:
# Feature flag
FF_KITS_ENABLED=true
# ActivePieces integration
ACTIVEPIECES_API_URL=http://localhost:8080
ACTIVEPIECES_API_KEY=your_key_here
# Credential encryption
KIT_CREDENTIAL_ENCRYPTION_KEY=your_32_byte_hex_keyGenerate encryption key:
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"Sample Kits
Included Kits:
-
Claude Code Kit (
src/kits/packages/claude-code/)- Generates
CLAUDE.mdfiles - Type: file
- Operations: spec, plan, validate
- Generates
-
HubSpot CRM Kit (
src/kits/packages/hubspot-crm/)- Configures HubSpot pipelines and properties
- Type: platform
- Operations: spec, check, discover, plan, validate, apply, verify
Additional Resources
Technical Specifications:
- Kit Package Specification:
docs/kits/kit-package-v1-specification.md - Kit Protocol:
docs/kits/kit-protocol-specification.md - Testing Framework:
docs/kits/kit-testing-framework.md - Self-Recovery:
docs/kits/kit-self-recovery-specification.md - Kit Factory:
docs/kits/kit-knowledge-layer-and-factory.md
Authoring Guide:
- Kit Authoring:
docs/kits/kit-authoring-guide.md
Test Suite:
- 901 total tests
- 98.3% pass rate (886 passing)
- Location:
tests/unit/kits/
Graph-Driven Execution
In the automation-consolidation branch, Kits execute as part of the graph-driven generation pipeline. BatchCoordinator (graph-engine/src/generation/batch-coordinator.ts) orchestrates the full flow; Kits are responsible for executing individual impl graph nodes.
KitJobInput Protocol
Kits receive a KitJobInput struct:
interface KitJobInput {
specNode: ContextArtifact; // the spec node to implement
implNode: ContextArtifact; // the target impl node
upstreamContracts: InterfaceContract[]; // what upstream modules expose
patternSeed?: PatternMatch; // best-matching pattern (if any)
structuralDiff?: StructuralDiff; // CEGIS feedback from prior attempt
projectContext: ProjectContext; // tech stack, conventions, config
}Kits return a KitJobResult:
interface KitJobResult {
virtualTree: VirtualTree; // generated file tree (in-memory)
interfaceContract: InterfaceContract; // what this module exposes downstream
patternContribution?: PatternCandidate; // novel pattern worth extracting
}CEGIS Retry Loop
BatchCoordinator wraps each Kit execution with up to 3 CEGIS retries:
- Kit generates output (
structuralDiffis null on attempt 1) - Output verified against spec node requirements
- If fail:
StructuralDiff(missing/extra/mismatched elements) is computed and fed back asstructuralDiffon next attempt - After 3 failures: node marked
FAILED, generation continues
Interface Contracts
Each Kit execution produces an InterfaceContract stored in the impl node's graph metadata. Downstream Kits receive upstream contracts via loadUpstreamContracts() — enabling compatible generated modules without guessing about neighbors.
Pattern Seeds
If PatternMatcher finds a high-scoring pattern for the impl node, it is passed as patternSeed. Kits should use the pattern as a structural skeleton and adapt it to the specific spec node's metadata.
ContainerPlatform
Kits execute as container jobs via RailwayContainerPlatform — a PostgreSQL-backed job queue with SELECT FOR UPDATE SKIP LOCKED for distributed Kit workers.
Last Updated: March 2026