Skip to main content
POST
/
evaluate
Evaluate datasets with specified metrics
curl --request POST \
  --url https://beta.getplum.ai/v1/evaluate \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --data '{
  "synthetic_data_id": "<string>",
  "seed_data_id": "<string>",
  "metrics_id": "<string>",
  "pair_query": {
    "latest_n_pairs": 123,
    "pair_labels": [
      "<string>"
    ],
    "last_n_seconds": 123
  }
}'
{
  "eval_results_id": "<string>",
  "dataset_id": "<string>",
  "created_at": "<string>",
  "scores": [
    {
      "metric": "<string>",
      "mean_score": 123,
      "std_dev": 123,
      "ci_low": 123,
      "ci_high": 123,
      "ci_confidence": 123,
      "median_score": 123,
      "min_score": 123,
      "max_score": 123,
      "lowest_scoring_pairs": [
        {
          "pair_id": "<string>",
          "score_reason": "<string>"
        }
      ]
    }
  ],
  "pair_count": 123
}

Authorizations

Authorization
string
header
required

Body

application/json
synthetic_data_id
string

ID of synthetic dataset to evaluate (either this or seed_data_id is required)

seed_data_id
string

ID of seed dataset to evaluate (either this or synthetic_data_id is required)

metrics_id
string

ID of metrics to use for evaluation (will use latest if not provided)

pair_query
object

Optional query parameters to filter dataset pairs

Response

Evaluation successfully completed

eval_results_id
string
dataset_id
string
created_at
string
scores
object[]
pair_count
integer
I