Skip to main content

​
Get an access key

https://auth.getplum.ai Sign up here to get a Plum AI API key.

​
Call the Plum AI - API

Instead of running curl commands, you can also use the API via the Postman interface. We recommend Postman over curl.

​
Step 1: Upload your data

All you need to get started is:
  1. The system prompt you’ve been using
  2. User prompts (inputs) you’ve sent to the LLM
  3. Outputs that the LLM has returned
curl -H "Authorization: YOUR_KEY_HERE" -d '{"data":[{"input":"THE FIRST USER PROMPT HERE", "output":"THE FIRST LLM RESPONSE HERE"},
{"input":"ANOTHER USER PROMPT HERE", "output":"THE SECOND LLM RESPONSE HERE"}], "system_prompt":"YOUR SYSTEM PROMPT HERE"}' beta.getplum.ai/v1/data/seed
You will get a dataset ID in the response. Keep it somewhere safe because you’ll need it later.

​
Step 2: Generate metric definitions based on your system prompt

The ability to evaluate your model comes from well-defined metrics. Plum creates criteria based on your business use case, which it then uses to score the model. 
curl -H "Authorization: YOUR_KEY_HERE" -d '{"system_prompt":"YOUR SYSTEM PROMPT GOES HERE"}' beta.getplum.ai/v1/questions
Keep the metrics ID in the response somewhere safe. You’ll need it in the next step.

​
Step 3: Generate evaluation results based on your uploaded data

In this step, Plum returns a score for each of the criteria defined in the previous step. These scores are out of a maximum of 5. These evaluation results will be used in the next step. Plum’s goal is to boost the lowest of these scores. 
curl -H "Authorization: YOUR_KEY_HERE" -d '{"prompt":"YOUR SYSTEM PROMPT GOES HERE", 
"seed_data_id":"YOUR SEED DATA ID FROM STEP 1 GOES HERE", 
"metrics_id":"YOUR METRICS ID FROM STEP 2 GOES HERE"}' beta.getplum.ai/v1/evaluate

​
Step 4: Generate synthetic data based on your uploaded seed data and evaluation results

This step generates synthetic inputs and outputs, each pair being a variation of one of the seed pairs you uploaded in Step 1. Adjust the multiple to adjust the size of the dataset you would like to generate. For each example in your seed dataset, Plum will generate multiple synthetic examples similar to it. You will need at least 50-100 examples, so set the multiple accordingly. 
curl -H "Authorization: YOUR_KEY_HERE" -d '{
"multiple":2, 
"seed_data_id":"YOUR SEED DATA ID GOES HERE", 
"eval_results_id":"YOUR EVAL RESULTS ID FROM STEP 3 GOES HERE"}' beta.getplum.ai/v1/augment

​
Step 5: Download the synthetic data in a format that OpenAI’s fine-tuning UI accepts

OpenAI provides the ability to fine-tune and customize their latest models. In order to do this, you will use the synthetic data that Plum generated in the previous step. 
curl -H "Authorization: YOUR_KEY_HERE" beta.getplum.ai/v1/data/synthetic_openai/1729625334491 > train.jsonl

​
Step 6: Upload the synthetic data to a major LLM provider like OpenAI’s fine-tuning API

Once you have a train.jsonl from Step 5, you could optionally create another file, validation.jsonl, using real heldout data that you haven’t used in the seed dataset.
  1. Go to the OpenAI fine-tuning page: https://platform.openai.com/finetune
  2. Click “Create”.
  3. Upload new training data.
  4. Optional: Select “Upload new” validation data. Drag your validation .jsonl file to upload.
After around 15 minutes, the fine-tuning run completes, and OpenAI will provide a customized model ID that you can start using. Congratulations! You’ve completed one round of fine-tuning using Plum AI. Unlock your data flywheel: generate a new set of data using your fine-tuned model, create synthetic data using Plum AI, and start another round of fine-tuning.
⌘I