Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updated python and wolfram examples

...

Textstyles
Colorrgb(236,0,140)

DRAFT SPEC |Services are in active development and are subject to change Services not yet live


The Charity Engine Smart Proxy Post Processing (SP3) interface allows web data that is collected via Smart Proxy scripts to be processed by any application or execution environment available or supported by the Charity Engine network. This processing takes place on the same node that collects the data.

...

To submit data for local processing on the proxy node, include reference the autoloaded "charityengine" module in a Node.js module in a smart proxy script, and call the desired processing function . (See *see Section 2 below for definitions of specific functions).

Processing functions return a Promise object, which resolves with the response from the model or application. This By using await, this response can then be used as appropriate within the Smart Proxy script and can be included in the crawl output.

...

Code Block
languagetext
charityengine.embeddings('llama3.2:3b', textCollectedFromCrawl)
  .then(response => console.log('Embedding response:', response))
  .catch(error => console.error('Error:', error));

To run a Docker container to calculate a mathematical sum using a custom script named "calc-sum.js":

Code Block
languagetext
charityengine.docker('node:slim', 'node /local/input/calc-sum.js 1 2 3 4.2', 'calc-sum.js')
  .then(response => console.log('The sum is:', response))
  .catch(error => console.error('Error:', error));;

To include multiple outputs, the responses from crawls and functions can Instead of using console output, which is useful for debugging locally, the response could be included in an object and returned from the Smart Proxy script so that it would be added to , as the return value defines the job output.

Processing Functions

...

Any image that can be pulled from Docker Hub, or that is publicly accessible on the web, can be used for post-processing by calling the docker() function (*subject to specs of instance types used):

Code Block
languagejs
charityengine.docker(image, commandline, inputfile)
PARAMETERS
  image // NameURL or Docker Hub name of the Docker image to run [string] [required]
  commandline // Command to execute within the container [string] [required]
  inputfile // Names of local files to use as input [array of string]

Execution Environments

The default execution environment is Node.js, per Section 1 above, "General Usage". For running source code or scripts in other interpreted languages, functions are available for named execution environments.

...

just download the execution environment of your choice from Docker Hub, following the instructions in Section 2.1 above. For example:

Code Block
languagejs
charityengine.python(commandline, inputfile)
PARAMETERS
  commandline // Command line for the application [string] [required]
  inputfile // Names of local files to use as input [array of string]docker("python:slim", "python /local/input/hello-world.py", ["hello-world.py"])

Charity Engine Application Library

...

For example, to generate a chi-squared distribution with v with 22 degrees of freedom using Wolfram Engine:

Code Block
languagetext
charityengine.wolframengine('ChiSquareDistribution[v]')
  .then(response => console.log('Response:', response))
  .catch(error => console.error('Error:', error)22]');

vina

Run a protein-ligand binding simulation

...