Authorizations
Body
application/json
String description of each metric
API Reference
- GETList all seed datasets
- GETRetrieve a specific seed dataset by ID
- POSTInsert or overwrite seed data by ID
- POSTAdd a single pair to a dataset (creates new dataset if the system_prompt_template that's provided doesn't correspond to an existing dataset)
- POSTAdd a single pair to an existing seed dataset by ID
- POSTUpload a new seed dataset
- GETList all synthetic datasets
- GETRetrieve synthetic data by ID
- PUTInsert or overwrite synthetic data by ID
- POSTUpload new synthetic data
- GETRetrieve synthetic data formatted for OpenAI by ID
- GETGet a specific metric definition by ID
- GETGet a specific evaluation result by ID
- GETExport evaluation results as TSV file
- PUTUpdate evaluation result with human feedback
- POSTGenerate evaluation metrics
- GETList all available evaluation metrics
- GETList all available evaluation results
- POSTCreate custom metric definitions
- POSTEvaluate datasets with specified metrics
- POSTAugment seed data to generate synthetic data
- GETGather high-scoring and positively critiqued pairs from evaluation results
- GETGather high-scoring and positively critiqued pairs formatted for OpenAI fine-tuning
- POSTCompare multiple prompts against a dataset
- POSTGenerate an improved prompt based on comparison results
Create custom metric definitions
Copy
curl --request POST \
--url https://beta.getplum.ai/v1/specify_questions \
--header 'Authorization: <api-key>' \
--header 'Content-Type: application/json' \
--data '{
"metrics": [
"<string>"
]
}'Copy
{
"metrics_id": "<string>",
"created_at": "<string>"
}API Reference
Create custom metric definitions
POST
/
specify_questions
Create custom metric definitions
Copy
curl --request POST \
--url https://beta.getplum.ai/v1/specify_questions \
--header 'Authorization: <api-key>' \
--header 'Content-Type: application/json' \
--data '{
"metrics": [
"<string>"
]
}'Copy
{
"metrics_id": "<string>",
"created_at": "<string>"
}⌘I