Data Fetching

ObjectWeaver allows for making external requests within the schema definitions to enhance data generation. This is done using the Req field within a property definition.

Adding External Requests

• Req Field: Define the external request within the Req field of the property definition
• Headers: Include any additional headers needed for the request in the Headers map. It is best practice to always include an authorization token
• Custom Body: Insert a custom body containing necessary data, such as the ID of the MeiliSearch collection, to facilitate efficient database searches
• Select Fields (Optional): Assign fields to be sent out with the request using the Fields property within the NarrowFocus section
• Processing Order (Optional): Ensure the relevant data is available before sending the request by using the ProcessingOrder functionality

Example: Page Definition with External Request

Below is an example schema definition that demonstrates how to incorporate an external request:

Go

pageDefinition := jsonSchema.Definition{
    Type:            jsonSchema.Object,
    Instruction:     "Create the details and information for a single page within a book.",
    ProcessingOrder: []string{"outline"},
    Model:           jsonSchema.ClaudeSonnet,

    Properties: map[string]jsonSchema.Definition{
        "outline": {
            Type:         jsonSchema.String,
            Instruction:  "Create a detailed overview of what content this page should include. This page outline should be detailed enough to ensure that it logically leads to the next page, creating a satisfying narrative progression. This outline should be a minimum of 250 words.",
            SystemPrompt: &outlineSystem,
        },
        "finalContent": {
            Type:         jsonSchema.String,
            SystemPrompt: &systemPrompt,
            Req: &jsonSchema.RequestFormat{
                URL:     "http://interactions-store:8001/getMeiliData",
                Method:  jsonSchema.POST,
                Headers: map[string]string{
                    "Authorization": fmt.Sprintf("Bearer %s", os.Getenv("OBJECT-GEN-KEY")),
                    // Add any additional headers here
                },
                Body: map[string]interface{}{
                    "projectId": projectId,
                },
                //To select the information that will be sent out
                RequiredFields: []string{"outline"}
            },
            NarrowFocus: &jsonSchema.Focus{
                Fields: []string{"outline"},
                Prompt: instruct,
            },
        },
    },
}

Python

page_definition = Definition(
    definition_type="object",
    instruction="Create the details and information for a single page within a book.",
    processing_order=["outline"],
    model="claude-sonnet",
    properties={
        "outline": Definition(
            definition_type="string",
            instruction="Create a detailed overview of what content this page should include. This page outline should be detailed enough to ensure that it logically leads to the next page, creating a satisfying narrative progression. This outline should be a minimum of 250 words.",
            system_prompt=outline_system
        ),
        "finalContent": Definition(
            definition_type="string",
            system_prompt=system_prompt,
            req=RequestFormat(
                url="http://interactions-store:8001/getMeiliData",
                method="POST",
                headers={
                    "Authorization": f"Bearer {os.getenv('OBJECT-GEN-KEY')}",
                    # Add any additional headers here
                },
                body={
                    "projectId": project_id
                },
                # To select the information that will be sent out
                required_fields=["outline"]
            ),
            narrow_focus=Focus(
                fields=["outline"],
                prompt=instruct
            )
        )
    }
)

JavaScript

const pageDefinition = {
    type: "object",
    instruction: "Create the details and information for a single page within a book.",
    processingOrder: ["outline"],
    model: "claude-sonnet",
    properties: {
        outline: {
            type: "string",
            instruction: "Create a detailed overview of what content this page should include. This page outline should be detailed enough to ensure that it logically leads to the next page, creating a satisfying narrative progression. This outline should be a minimum of 250 words.",
            systemPrompt: outlineSystem
        },
        finalContent: {
            type: "string",
            systemPrompt: systemPrompt,
            req: {
                url: "http://interactions-store:8001/getMeiliData",
                method: "POST",
                headers: {
                    "Authorization": `Bearer ${process.env.OBJECT_GEN_KEY}`,
                    // Add any additional headers here
                },
                body: {
                    projectId: projectId
                },
                // To select the information that will be sent out
                requiredFields: ["outline"]
            },
            narrowFocus: {
                fields: ["outline"],
                prompt: instruct
            }
        }
    }
};

Best Practices

1. Authorization Token: Always include an authorization token in the headers to secure the request
2. Custom Body: Use a custom body to pass necessary identifiers and other relevant data
3. Select Fields: Specify fields that should be sent with the request to ensure all required data is included
4. Processing Order: Use the ProcessingOrder to sequence the generation process, ensuring prerequisite data is available before the request is made

By following these guidelines, you can efficiently integrate external data sources into your schema definitions, ensuring robust and accurate data generation.