Web sdk
User Feedbacks

Feedback

With every trace called, we keep a snapshot of user interaction on client side so user can rewind and get more context when providing feedback.

Here is how you can collect user feedback on client side with Agiflow sdk.

Inline feedback

You can use actionId generated with track or trackAsync to identify user feedback such as thumb up / thumb down directly on client with below snippet. Additionally, you can associate a backend trace with your unique identifier (workflow Id, message Id, etc...) and use that to provide feedback:

agiflowClient.reportScore(
  actionId, // ActionId is generated from track
  0.6 // Normalised score
);

We use normalised score to make it easier for fine-tune. Here is a reference for conversion:

  • Thumbs up = 1; thumbs down = 0;
  • Rank 0 -> 10 is converted to [0, 1].

Feedback widget

Feedback widget is a powerful tool to provide feedback on the AI output and increase the transparency of your AI application. There are two modes you can display feedback widget at this stage.

Simple chat feedback

const agiflowClient = new Agiflow();
agiflowClient.init(
    process.env.CLIENT_KEY, // Agiflow client key (please don't use API key here)
    feedbackPreviews: ['screen'],
);
 
agiflowClient.openWidget();

When user agree to provide feedback, this will replay the screen state when the AI interaction happens.

Complex workflow feedback

const agiflowClient = new Agiflow();
agiflowClient.init(
    process.env.CLIENT_KEY, // Agiflow client key (please don't use API key here)
    feedbackPreviews: ['workflow'],
);
 
agiflowClient.openWidget();

Unlike simple chat feedback when there is only one AI output corresponding to single user input, a complex workflow involes multiple AI agents. Simply provide correction on the output is not helpful as it can't be used for evaluating AI prompts or fine-tuning.

This mode displays a workflow chart where your end-user can click on particular component of AI workflow to review output and provide feedback. This works best for internal tools or B2B businesses.

Subscribe to issues detected by guardrails

If you wish to only open feedback widget when there are issues with LLM, you can subscribe to new issues and decide to call openWidget at any time liked below:

agiflowClient.subscribeToNewIssue((issue) => {
    ...
});
TracingUser Identifier