...
Textstyles | ||
---|---|---|
| ||
DRAFT SPEC |Services are in active development and are subject to change Services not yet live |
The Charity Engine Smart Proxy Post Processing (SP3) interface allows web data that is collected via Smart Proxy scripts to be processed by any application or execution environment available or supported by the Charity Engine network. This processing takes place on the same node that collects the data.
...
To submit data for local processing on the proxy node, include reference the autoloaded "charityengine" module in a Node.js module in a smart proxy script, and call the desired processing function . (See *see Section 2 below for definitions of specific functions).
Processing functions return a Promise object, which resolves with the response from the model or application. This By using await
, this response can then be used as appropriate within the Smart Proxy script and can be included in the crawl output.
...
Code Block | ||
---|---|---|
| ||
charityengine.embeddings('llama3.2:3b', textCollectedFromCrawl)
.then(response => console.log('Embedding response:', response))
.catch(error => console.error('Error:', error)); |
To run a Docker container to calculate a mathematical sum using a custom script named "calc-sum.js":
Code Block | ||
---|---|---|
| ||
charityengine.docker('node:slim', 'node /local/input/calc-sum.js 1 2 3 4.2', 'calc-sum.js') .then(response => console.log('The sum is:', response)) .catch(error => console.error('Error:', error));; |
To include multiple outputs, the responses from crawls and functions can Instead of using console output, which is useful for debugging locally, the response could be included in an object and returned from the Smart Proxy script so that it would be added to , as the return value defines the job output.
Processing Functions
...
Any image that can be pulled from Docker Hub, or that is publicly accessible on the web, can be used for post-processing by calling the docker()
function (*subject to specs of instance types used):
Code Block | ||
---|---|---|
| ||
charityengine.docker(image, commandline, inputfile) PARAMETERS image // Name URL or Docker Hub name of the Docker image to run [string] [required] commandline // Command to execute within the container [string] [required] inputfile // Names of local files to use as input [array of string] |
Execution Environments
The default execution environment is Node.js, per Section 1 above, "General Usage". For running source code or scripts in other interpreted languages, functions are available for named execution environments.
...
just download the execution environment of your choice from Docker Hub, following the instructions in Section 2.1 above. For example:
Code Block | ||
---|---|---|
| ||
charityengine.python(commandline, inputfile) PARAMETERS commandline // Command line for the application [string] [required] inputfile // Names of local files to use as input [array of string]docker("python:slim", "python /local/input/hello-world.py", ["hello-world.py"]) |
Charity Engine Application Library
...
Note that some of these functions make use of proprietary software, which could incur additional runtime charges.
inference
Run LLM inference on a string
Code Block | ||
---|---|---|
| ||
charityengine.inference(model, prompt, assets, context, system, template, options) PARAMETERS model // Name of the model to use [string] [required] prompt // Text to pass to the model as input [string] [required] assets // Images or files [array of string] context // Additional context for the model [string] system // System parameters for the model [string] template // Template to guide response format [string] options // Additional options for the request [object] |
embeddings
Generate a vector embedding for a string
Code Block | ||
---|---|---|
| ||
charityengine.embeddings(model, prompt) PARAMETERS model // Name of the model to use [string] [required] prompt // Text to pass to the model as input [string] [required] |
wolframengine
Run Wolfram Language Code
Code Block | ||
---|---|---|
| ||
charityengine.wolframengine(commandline, inputfile) PARAMETERS commandline // Command to execute within the application [string] [required] inputfile // Names of local files to use as input [array of string] |
To For example, to generate a chi-squared distribution with v
22
degrees of freedom using Wolfram Engine:
Code Block | ||
---|---|---|
| ||
charityengine.wolframengine('ChiSquareDistribution[v]')
.then(response => console.log('Response:', response))
.catch(error => console.error('Error:', error)); |
...
22]'); |
vina
Run a protein-ligand binding simulation
Code Block | ||
---|---|---|
| ||
charityengine.vina(commandline, inputfile) PARAMETERS commandline // Command line for the application [string] [required] inputfile // Names of local files to use as input [array of string] |
blastp
Compare an amino acid sequence (protein sequence) against a protein sequence database
Code Block | ||
---|---|---|
| ||
charityengine.blastp(commandline, inputfile) PARAMETERS commandline // Command line for the application [string] [required] inputfile // Names of local files to use as input [array of string] |
...