...
Textstyles | ||
---|---|---|
| ||
DRAFT SPEC | Services not yet live |
The Charity Engine Smart Proxy Post Processing (SP3) interface allows web data that is collected via Smart Proxy scripts to be processed by any application or execution environment available or supported by the Charity Engine network. This processing takes place on the same node that collects the data.
...
To submit data for local processing on the proxy node, include reference the autoloaded "charityengine" module in a Node.js module in a smart proxy script, and call the desired processing function . (See *see Section 2 below for definitions of specific functions).
Processing functions return a Promise object, which resolves with the response from the model or application. This By using await
, this response can then be used as appropriate within the Smart Proxy script and can be included in the crawl output.
...
Code Block | ||
---|---|---|
| ||
charityengine.embeddings('llama3.2:3b', textCollectedFromCrawl) .then(response => console.log('Embedding response:', response)) .catch(error => console.error('Error:', error)); |
To run a Docker container to calculate a mathematical sum using a custom script named "calc-sum.js":
Code Block | ||
---|---|---|
| ||
charityengine.docker('node:slim', 'node /local/input/calc-sum.js 1 2 3 4.2', 'calc-sum.js') .then(response => console.log('The sum is:', response)) .catch(error => console.error('Error:', error));; |
To include multiple outputs, the responses from crawls and functions can Instead of using console output, which is useful for debugging locally, the response could be included in an object and returned from the Smart Proxy script so that it would be added to , as the return value defines the job output.
Processing Functions
...
Any image that can be pulled from Docker Hub, or that is publicly accessible on the web, can be used for post-processing by calling the docker()
function (*subject to specs of instance types used):
Code Block | ||
---|---|---|
| ||
charityengine.docker(image, commandline, inputfile) PARAMETERS image // Name URL or Docker Hub name of the Docker image to run [string] [required] commandline // Command to execute within the container [string] [required] inputfile // Names of local files to use as input [array of string] |
Execution Environments
The default execution environment is Node.js, per Section 1 above, "General Usage". For running source code or scripts in other interpreted languages, functions are available for named execution environments.
...
just download the execution environment of your choice from Docker Hub, following the instructions in Section 2.1 above. For example:
Code Block | ||
---|---|---|
| ||
charityengine.python(commandline, inputfile) PARAMETERS commandline // Command line for the application [string] [required] inputfile // Names of local files to use as input [array of string]docker("python:slim", "python /local/input/hello-world.py", ["hello-world.py"]) |
Charity Engine Application Library
...
For example, to generate a chi-squared distribution with v
with 22
degrees of freedom using Wolfram Engine:
Code Block | ||
---|---|---|
| ||
charityengine.wolframengine('ChiSquareDistribution[v]') .then(response => console.log('Response:', response)) .catch(error => console.error('Error:', error)22]'); |
vina
Run a protein-ligand binding simulation
...