Search…
API
How to inference GPT-J with the HTTP API.

Generate completion

post
https://example.forefront.link
Generate Completion

Example request

cURL
Python
1
curl https://DEPLOYMENT_NAME-TEAM_SLUG.forefront.link \
2
-H 'Content-Type: application/json' \
3
-H 'Authorization: Bearer YOUR_API_KEY' \
4
-d '{
5
"text": "Once upon a time",
6
"top_p": 1,
7
"top_k": 40,
8
"temperature": 0.8,
9
"repetition_penalty": 1.2,
10
"length": 256,
11
"stop_sequences": ["bird", "cat", "dog"]
12
}'
Copied!
1
import requests
2
3
headers = {"Authorization": "Bearer <INSERT_YOUR_TOKEN>"}
4
5
data = {
6
"text": "Once upon a time",
7
"top_p": 1,
8
"top_k": 40,
9
"temperature": 0.8,
10
"repetition_penalty": 1.2,
11
"length": 256,
12
"stop_sequences": ["bird", "cat", "dog"]
13
}
14
15
res = requests.post(
16
"https://vanilla-gpt-j-forefront.forefront.link",
17
json=data,
18
headers=headers
19
)
Copied!

Stream completion

get
https://example.forefront.link
Stream completion with server sent events

How to use

Our streaming API uses the built in Javascript EventSource object, streaming tokens via Server Sent Events (SSE). If you're using another language, you'll need to find an equivalent API to listen for these events. Because the EventSource API doesn't support POST requests, you'll have to put all of your params into a query string, and URI Encode it appropriately. Here's some sample front end JS code showing how you can do this, which you can use as a starting point for other languages.
1
url = '' // for example <https://example.forefront.link>
2
api_key = '' // navigate to Settings => API Key
3
4
encodeURIComponent(JSON.stringify(stop.map(s => s.replaceAll('\\n', '\n'))))
5
)
6
7
// helper function to construct URL
8
const makeUrlForEventSource = (input) => (
9
`${url}/?text=${encodeURIComponent(input)
10
}&length=${length
11
}&repetition_penalty=${rep
12
}&temperature=${temperature
13
}&top_p=${topP
14
}&top_k=${topK
15
}&stop_sequences=${processStopSequences(stop)
16
}&authorization=Bearer ${api_key}`
17
);
18
19
// helper function to submit request and listen for events
20
const handleEventSourceSubmit = (input) => {
21
22
const url = makeUrlForEventSource(input)
23
const source = new EventSource(url, { withCredentials: false });
24
25
source.addEventListener('update', (e) => {
26
handleStream(e.data)
27
})
28
29
source.addEventListener('end', () => {
30
source.close()
31
})
32
33
}
Copied!

Retrieve API Key

To retrieve your API key, navigate to Settings > API Key.

Retrieve Model URL

To retrieve your model URL, navigate to the deployment and copy the URL under the model name.
Last modified 1mo ago