使用cloudflare免费搭建自己几十个开源大模型api接口
cloudflare推出了自己的免费大模型在线运行和api环境Workers AI LLM Playground,主持几十种主流开源大模型,不仅可以免费在线使用,还提供了在cloudflare中可运行调用这些大模型api的代码,简直就是白嫖啊。
web免费使用
免费在线使用地址:https://playground.ai.cloudflare.com/
api 调用
有两种调用方式1、cloudflare官方api调用
需要申请api key,申请key地址::https://dash.cloudflare.com/profile/api-tokens
申请完后就能直接通过curl、fetch、python来调用了
下面是python代码
#!/usr/local/python3/bin/python3 # -*- coding: utf-8 -* import requests #apikey申请地址:https://dash.cloudflare.com/profile/api-tokens API_TOKEN=""; API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/fbf170d7957b43197af922e09cb9b936/ai/run/" headers = {"Authorization": "Bearer {API_TOKEN}"} def run(model, inputs): input = { "messages": inputs } response = requests.post(f"{API_BASE_URL}{model}", headers=headers, json=input) return response.json() inputs = [ { "role": "system", "content": "You are a friendly assistan that helps write stories" }, { "role": "user", "content": "Write a short story about a llama that goes on a journey to find an orange cloud "} ]; output = run("@cf/meta/llama-2-7b-chat-int8", inputs) print(output)2、自己部署worker ai
登录你的cloudflare后台,选择worker ai
然后创建一个worker,选择llm app这个模板
然后最下角点击部署按钮即可
完整的worker代码
import { Ai } from './vendor/@cloudflare/ai.js'; export default { async fetch(request, env) { const tasks = []; const ai = new Ai(env.AI); // prompt - simple completion style input let simple = { prompt: 'Tell me a joke about Cloudflare' }; let response = await ai.run('@cf/meta/llama-3-8b-instruct', simple); tasks.push({ inputs: simple, response }); // messages - chat style input let chat = { messages: [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'Who won the world series in 2020?' } ] }; response = await ai.run('@cf/meta/llama-3-8b-instruct', chat); tasks.push({ inputs: chat, response }); return Response.json(tasks); } };
创建好ai worker后就有一个域名可以访问了,可以在setting-》trigger中绑定自己的域名。
ok,这样就能搭建自己的worker api来免费调用像llama3几十种的开源ai大模型了。
网友评论0