Skip to main content

Decision Points System

Overview​

The Decision Points system enables conditional branching in AI generation workflows based on LLM evaluation scores or generated field values. This allows you to build complex, intelligent decision trees where the quality or content of generated outputs determines the next generation step.

Core Concepts​

1. ScoringCriteria - Evaluation Metrics​

Define how to evaluate the quality of generated content across multiple dimensions.

ScoringCriteria: &ScoringCriteria{
    Dimensions: map[string]ScoringDimension{
        "technical_accuracy": {
            Description: "Rate technical accuracy from 0-100",
            Scale: &ScoreScale{Min: 0, Max: 100},
            Type: ScoreNumeric,
            Weight: 0.6,
        },
        "readability": {
            Description: "Rate readability from 0-100",
            Scale: &ScoreScale{Min: 0, Max: 100},
            Type: ScoreNumeric,
            Weight: 0.4,
        },
    },
    AggregationMethod: AggregateWeightedAverage,
}

Score Types:

  • ScoreNumeric - Numeric scores within a range (e.g., 0-100)
  • ScoreBoolean - Binary true/false evaluation
  • ScoreCategorical - Categorical values (e.g., "low", "medium", "high")

Aggregation Methods:

  • AggregateWeightedAverage - Weighted average based on dimension weights
  • AggregateMinimum - Use the lowest score (all must be good)
  • AggregateMaximum - Use the highest score (any can be good)

2. DecisionPoint - Conditional Routing​

Route to different Definitions based on conditions.

DecisionPoint: &DecisionPoint{
    Name: "QualityRouter",
    Strategy: RouteByScore,
    Branches: []ConditionalBranch{
        {
            Name: "HighAccuracyLowReadability",
            Conditions: []Condition{
                {Field: "technical_accuracy", Operator: OpGreaterThan, Value: 70},
                {Field: "readability", Operator: OpLessThan, Value: 60},
            },
            Then: Definition{
                Type: String,
                Instruction: "Simplify language while maintaining accuracy",
                SelectFields: []string{"content"}, // Must be in first layer
            },
        },
    },
    Default: &Definition{...},
}
SelectFields Limitation

When using SelectFields in a decision branch's Then definition, you can only reference fields from the first layer of the parent object. Nested fields cannot be selected and will be lost if not included at the root level.

Routing Strategies:

  • RouteByScore - Use LLM evaluation scores
  • RouteByField - Use generated field values (via SelectFields)
  • RouteByHybrid - Combine both score and field conditions

Comparison Operators:

  • OpEqual (==), OpNotEqual (!=)
  • OpGreaterThan (>), OpLessThan (<)
  • OpGreaterThanOrEqual (>=), OpLessThanOrEqual (<=)
  • OpIn (value in list), OpNotIn (value not in list)
  • OpContains (string contains substring)

3. RecursiveLoop - Iterative Refinement​

Generate content multiple times and select the best result.

RecursiveLoop: &RecursiveLoop{
    MaxIterations: 5,
    Selection: SelectHighestScore,
    TerminateWhen: []TerminationCondition{
        {
            Field: "quality",
            Operator: OpGreaterThanOrEqual,
            Value: 85,
        },
    },
    FeedbackPrompt: "Improve based on previous attempt",
    IncludePreviousAttempts: true,
}

Selection Strategies:

  • SelectHighestScore - Keep the best-scoring iteration
  • SelectLowestScore - Keep the worst-scoring iteration
  • SelectLatest - Keep the most recent iteration
  • SelectFirst - Keep the first successful iteration
  • SelectAll - Return all iterations as an array

Usage Patterns​

Pattern 1: Score-Based Quality Control​

Use Case: Generate content, evaluate quality, and route to improvement steps if needed.

Postman/REST API Example:

{
  "prompt": "Generate a technical blog post about microservices architecture and best practices",
  "definition": {
    "type": "object",
    "properties": {
      "content": {
        "type": "string",
        "instruction": "Generate a technical blog post about microservices",
        "scoringCriteria": {
          "dimensions": {
            "technical_accuracy": {
              "description": "Rate the technical accuracy from 0-100",
              "type": "numeric"
            },
            "readability": {
              "description": "Rate the readability from 0-100",
              "type": "numeric"
            }
          }
        },
        "decisionPoint": {
          "name": "ContentQualityRouter",
          "strategy": "score",
          "branches": [
            {
              "name": "HighAccuracyLowReadability",
              "conditions": [
                {
                  "field": "technical_accuracy",
                  "operator": "gt",
                  "number_value": 90
                },
                {
                  "field": "readability",
                  "operator": "lt",
                  "number_value": 90
                }
              ],
              "then": {
                "type": "object",
                "instruction": "Improve readability while maintaining technical accuracy",
                "selectFields": ["content"],
                "properties": {
                  "simplified_content": {
                    "type": "string",
                    "instruction": "Simplify the language"
                  }
                }
              }
            }
          ]
        }
      }
    }
  }
}

Go SDK Example:

Definition{
    Type: String,
    Instruction: "Generate technical blog post",
    
    ScoringCriteria: &ScoringCriteria{
        Dimensions: map[string]ScoringDimension{
            "quality": {
                Description: "Overall quality 0-100",
                Scale: &ScoreScale{Min: 0, Max: 100},
                Type: ScoreNumeric,
            },
        },
    },
    
    DecisionPoint: &DecisionPoint{
        Strategy: RouteByScore,
        Branches: []ConditionalBranch{
            {
                Name: "LowQuality",
                Conditions: []Condition{
                    {Field: "quality", Operator: OpLessThan, Value: 70},
                },
                Then: Definition{
                    Type: String,
                    Instruction: "Improve quality and add more detail",
                    SelectFields: []string{"content"},
                },
            },
            {
                Name: "HighQuality",
                Conditions: []Condition{
                    {Field: "quality", Operator: OpGreaterThanOrEqual, Value: 85},
                },
                Then: Definition{
                    Type: String,
                    Instruction: "Generate meta description and summary",
                },
            },
        },
    },
}
SelectFields in Decision Points

Note that selectFields: ["content"] references the content field from the first layer of the parent object. Fields nested deeper cannot be selected and will be lost.

Pattern 2: Field-Based Content Type Routing​

Use Case: Route to different generation strategies based on content type or requirements.

Definition{
    Type: Object,
    Properties: map[string]Definition{
        "requirements": {
            Type: Object,
            Properties: map[string]Definition{
                "is_technical": {Type: Boolean, Instruction: "Is technical?"},
                "audience": {Type: String, Instruction: "Beginner/Advanced?"},
            },
        },
        "content": {
            Type: String,
            SelectFields: []string{"requirements"},
            
            DecisionPoint: &DecisionPoint{
                Strategy: RouteByField,
                Branches: []ConditionalBranch{
                    {
                        Name: "TechnicalAdvanced",
                        Conditions: []Condition{
                            {Field: "is_technical", FieldPath: "requirements.is_technical", 
                             Operator: OpEqual, Value: true},
                            {Field: "audience", FieldPath: "requirements.audience", 
                             Operator: OpEqual, Value: "Advanced"},
                        },
                        Then: Definition{
                            Type: String,
                            Instruction: "Write advanced technical content with code",
                        },
                    },
                    {
                        Name: "NonTechnical",
                        Conditions: []Condition{
                            {Field: "is_technical", FieldPath: "requirements.is_technical", 
                             Operator: OpEqual, Value: false},
                        },
                        Then: Definition{
                            Type: String,
                            Instruction: "Write accessible content for general audience",
                        },
                    },
                },
            },
        },
    },
    ProcessingOrder: []string{"requirements", "content"},
}

Pattern 3: Hybrid Routing (Field + Score)​

Use Case: Complex decisions based on both content characteristics and quality metrics.

DecisionPoint: &DecisionPoint{
    Strategy: RouteByHybrid,
    Branches: []ConditionalBranch{
        {
            Name: "ComplexTopicLowQuality",
            Priority: 10, // Evaluate first
            Conditions: []Condition{
                // Field condition
                {Field: "complexity", FieldPath: "analysis.complexity", 
                 Operator: OpEqual, Value: "complex"},
                // Score condition
                {Field: "quality", Operator: OpLessThan, Value: 70},
            },
            Then: Definition{
                Type: String,
                Instruction: "Add detailed explanations and break down concepts",
            },
        },
        {
            Name: "SimpleTopicHighQuality",
            Conditions: []Condition{
                {Field: "complexity", FieldPath: "analysis.complexity", 
                 Operator: OpEqual, Value: "simple"},
                {Field: "quality", Operator: OpGreaterThanOrEqual, Value: 80},
            },
            Then: Definition{
                Type: String,
                Instruction: "Finalize and add engaging examples",
            },
        },
    },
}

Pattern 4: Iterative Refinement​

Use Case: Generate high-quality content through multiple attempts.

Definition{
    Type: String,
    Instruction: "Generate compelling product description",
    
    ScoringCriteria: &ScoringCriteria{
        Dimensions: map[string]ScoringDimension{
            "persuasiveness": {
                Description: "How persuasive? 0-100",
                Scale: &ScoreScale{Min: 0, Max: 100},
                Type: ScoreNumeric,
                Weight: 0.7,
            },
            "clarity": {
                Description: "How clear? 0-100",
                Scale: &ScoreScale{Min: 0, Max: 100},
                Type: ScoreNumeric,
                Weight: 0.3,
            },
        },
        AggregationMethod: AggregateWeightedAverage,
    },
    
    RecursiveLoop: &RecursiveLoop{
        MaxIterations: 5,
        Selection: SelectHighestScore, // Keep best attempt
        TerminateWhen: []TerminationCondition{
            {
                Field: "persuasiveness",
                Operator: OpGreaterThanOrEqual,
                Value: 90, // Stop early if persuasiveness >= 90
            },
        },
        FeedbackPrompt: "Improve by making more benefit-focused and compelling",
        IncludePreviousAttempts: true,
    },
}

Pattern 5: Nested Decision Trees​

Use Case: Multi-level decision trees for complex workflows.

Definition{
    Type: String,
    Instruction: "Generate draft",
    
    ScoringCriteria: &ScoringCriteria{
        Dimensions: map[string]ScoringDimension{
            "depth": {Description: "Content depth 0-100", Type: ScoreNumeric},
        },
    },
    
    DecisionPoint: &DecisionPoint{
        Name: "FirstLevelCheck",
        Strategy: RouteByScore,
        Branches: []ConditionalBranch{
            {
                Name: "ShallowContent",
                Conditions: []Condition{
                    {Field: "depth", Operator: OpLessThan, Value: 60},
                },
                Then: Definition{
                    Type: String,
                    Instruction: "Add more depth",
                    
                    ScoringCriteria: &ScoringCriteria{...},
                    
                    // Nested second-level decision
                    DecisionPoint: &DecisionPoint{
                        Name: "SecondLevelCheck",
                        Branches: []ConditionalBranch{
                            {
                                Name: "StillShallow",
                                Conditions: []Condition{
                                    {Field: "depth", Operator: OpLessThan, Value: 70},
                                },
                                Then: Definition{
                                    Type: String,
                                    Instruction: "Comprehensive rewrite",
                                },
                            },
                        },
                    },
                },
            },
        },
    },
}

Advanced Features​

ComplexSystem - Hierarchical Intelligence​

For production systems requiring supervision and intervention:

ComplexSystem{
    Name: "IntelligentContentGenerator",
    PrimarySystemPrompt: "You are an expert content creator...",
    
    RootSchema: Definition{...},
    
    MainThread: &MainThreadConfig{
        MonitoringStrategy: MonitorMilestone,
        InterventionRules: []InterventionRule{
            {
                Name: "LowQualityIntervention",
                Trigger: InterventionTrigger{
                    Type: TriggerLowScore,
                    ScoreThresholds: map[string]float64{"quality": 50.0},
                },
                Action: InterventionAction{
                    Type: ActionAugment,
                    FetchMemory: true,
                    OverridePrompt: stringPtr("Quality is low. Review examples and regenerate."),
                },
            },
        },
        HierarchicalDepth: 2,
    },
    
    Memory: &MemoryConfig{
        Enabled: true,
        SearchStrategy: SearchSemantic,
        RetrievalTriggers: []MemoryTrigger{
            {
                ScoreThreshold: &Condition{
                    Field: "quality",
                    Operator: OpLessThan,
                    Value: 60,
                },
                QueryTemplate: "Find high-quality examples of {topic}",
            },
        },
    },
}

Main Thread Features:

  • Monitoring Strategies:

    • MonitorContinuous - Real-time monitoring (expensive)
    • MonitorMilestone - Monitor at decision points (balanced)
    • MonitorFinal - Monitor only final output (minimal)

  • Intervention Actions:

    • ActionOverride - Replace current generation
    • ActionAugment - Add context and retry
    • ActionReset - Roll back to checkpoint
    • ActionTerminate - Stop processing
    • ActionEscalate - Send to parent thread

Lazy Config - Automatic Pattern Selection​

Let the system choose or create appropriate decision trees:

LazyConfig: &LazyConfigMode{
    Enabled: true,
    PatternLibrary: &PatternLibraryConfig{
        SourceURL: "https://patterns.objectweaver.ai/library",
        SelectionModel: "gpt-4",
        Tags: []string{"content_generation", "quality_control"},
    },
    CreationPrompt: "Analyze task and select appropriate pattern",
    SelfImprovement: &SelfImprovementConfig{
        Enabled: true,
        SaveSuccessfulPatterns: true,
        SuccessThreshold: 85.0,
        RefinementStrategy: RefineIterative,
        SafetyMechanisms: &SafetyConfig{
            RollbackOnFailure: true,
            PerformanceBaseline: map[string]float64{"quality": 70.0},
            MaxComplexity: 5,
        },
    },
    DeploymentMode: DeploySingleInstance,
}

Best Practices​

1. Start Simple, Iterate​

Begin with basic score-based routing:

DecisionPoint: &DecisionPoint{
    Strategy: RouteByScore,
    Branches: []ConditionalBranch{
        {
            Conditions: []Condition{
                {Field: "quality", Operator: OpLessThan, Value: 70},
            },
            Then: Definition{Instruction: "Improve quality"},
        },
    },
}

Then add complexity as needed.

2. Use Meaningful Score Descriptions​

Bad:

"quality": {Description: "Rate quality"}

Good:

"quality": {
    Description: "Rate overall quality from 0-100, where 0 is completely unusable and 100 is perfect. Consider accuracy, clarity, and completeness.",
    Scale: &ScoreScale{Min: 0, Max: 100},
}

3. Set Appropriate Thresholds​

Consider your use case:

  • Strict quality control: High thresholds (85-90)
  • Balanced approach: Medium thresholds (70-80)
  • Fast iteration: Lower thresholds (60-70)

4. Use Priority for Branch Ordering​

When multiple branches could match:

Branches: []ConditionalBranch{
    {
        Name: "SpecificCase",
        Priority: 10, // Check first
        Conditions: []Condition{...},
    },
    {
        Name: "GeneralCase",
        Priority: 5, // Check second
        Conditions: []Condition{...},
    },
}

5. Leverage SelectFields for Context​

Pass relevant context to improvement steps:

Then: Definition{
    Type: String,
    Instruction: "Improve content",
    SelectFields: []string{"original_content", "requirements", "feedback"},
}
SelectFields Constraint

Important: selectFields must reference fields in the first layer of the object. Fields nested deeper in the object hierarchy cannot be selected and will be lost. Ensure previously generated content you want to pass forward is at the root level of your definition.

Example:

{
  "definition": {
    "type": "object",
    "properties": {
      "content": {
        "type": "string",
        "instruction": "Generate a blog post",
        "decisionPoint": {
          "branches": [{
            "then": {
              "type": "object",
              "selectFields": ["content"],  // ✓ Valid - 'content' is in first layer
              "properties": {
                "improved_content": {
                  "type": "string",
                  "instruction": "Improve the content"
                }
              }
            }
          }]
        }
      }
    }
  }
}

6. Limit Recursive Depth​

Prevent infinite loops:

RecursiveLoop: &RecursiveLoop{
    MaxIterations: 5, // Hard limit
    TerminateWhen: []TerminationCondition{...}, // Soft exit
}

7. Use Hybrid Strategy for Complex Logic​

When decisions depend on multiple factors:

Strategy: RouteByHybrid,
Conditions: []Condition{
    {Field: "is_technical", FieldPath: "...", Operator: OpEqual, Value: true},
    {Field: "quality", Operator: OpLessThan, Value: 70},
    {Field: "complexity", FieldPath: "...", Operator: OpEqual, Value: "high"},
}

Common Patterns​

Quality Gate Pattern​

// Generate → Score → Route based on quality
ScoringCriteria → DecisionPoint → Branches

Iterative Improvement Pattern​

// Generate → Score → Regenerate until good enough
ScoringCriteria → RecursiveLoop → TerminateWhen

Type-Based Routing Pattern​

// Analyze type → Route to specialized generator
Field-based DecisionPoint → Type-specific Definitions

Hierarchical Supervision Pattern​

// Generate with oversight → Intervene if needed
ComplexSystem → MainThread → InterventionRules

Performance Considerations​

  1. Scoring Costs: Each score requires an LLM call. Use fewer dimensions for cost efficiency.
  2. Recursive Loops: Multiply costs by MaxIterations. Set reasonable limits.
  3. Main Thread: Continuous monitoring is expensive. Use milestone or final monitoring.
  4. Evaluation Models: Consider using cheaper models for scoring (e.g., GPT-3.5 vs GPT-4).

Examples​

See examples.go for complete working examples of:

  • Simple score-based routing
  • Field-based routing
  • Hybrid routing
  • Recursive loops
  • Nested decision trees
  • ComplexSystem with main thread
  • Lazy config and self-improvement

Integration with Existing Features​

The decision points system integrates seamlessly with:

  • ProcessingOrder: Control field evaluation sequence
  • SelectFields: Pass context between decision branches
  • SystemPrompt: Override prompts at any decision level
  • Model: Use different models for different branches
  • ImprovementProcess: Combine with existing improvement logic
  • Choices: Dynamic field selection before routing

Future Enhancements​

Planned features:

  • A/B testing of decision branches
  • Automatic threshold optimization
  • Decision tree visualization tools
  • Performance analytics and monitoring
  • Pattern sharing marketplace