End-to-end Observability

Agiflow aims to offer essential insights to companies regarding how end-users interact with AI products in a secure and compliant manner.

Framework overview

To ensure companies gain deeper product insights and obtain valuable, high-quality feedback from users, Agiflow employs a structured approach to capturing information, following this hierarchy:

Session

• By default, each session in Agiflow will generate an anonymous user. The end application has the option to associate user sessions with an alias.

Task

• A task is linked to the end-user's objective or intention. Task completion serves as a critical metric we utilise to gauge the usefulness of AI.

Action

• To complete a task, end-user would need to go through a series of actions.
• In the context of chat, it involves a series of questions and answers before the task is accomplished. Similarly, a user would undergo multiple image generation prompts to obtain the correct image they find useful.

Step (worflow, task, LLM calls)

• When user takes an action, it usually invoke an API endpoint and wait for results.
• Steps comprise a sequence of machine-to-machine communications within API invocation. The initial step commences with the user request, and the final step concludes with the API response.

User

• By default, Agiflow does not retain user tracking across sessions. However, for end-applications requiring insight into user behavior across sessions, they can associate users with anonymous sessions using an alias.

Distributed tracing

Agiflow adopts a distributed tracing approach to accurately capture telemetry information and provide companies with valuable data. Here's a brief overview of the data flow:

Client application

• When a user visits frontend application, a session is created by agiflowk client sdks.
• When a user interacts with application (provide input, clicking on button, etc...), a task is identified manually or automatically.
• A unique actionId is generated on the frontend and is sent to server using x-agiflow-trace-id header. This allows Agiflow to collaterate information from frontend to backend.

Server applications

• x-agiflow-trace-id is required to be forward to server applications to enable e2e tracing.
• Adopting Open Telemetry Span (opens in a new tab) method for tracing, Agiflow will associate user action with trace-id from the backend.
• The telemetry data collected from server is anomimised with PII mask before storing in Agiflow log database.

With this information, Agiflow can effectively give you the complete picture of user interaction with your AI features in a secured, scalable way.