Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Updated anchor link to use appropriate Confluence format

...

Code Block
languagejs
ce-cli [args]
OPTIONS
--version // Show version number [boolean]
--app // Application name (e.g. "charityengine:wolframengine"), docker image from Docker Hub (e.g. "docker:image-name") or a custom docker image URL (e.g. "docker:image-name https://example.com/file") [string] [required]
--auth // Authorization key [string] [required]
--batch // Arbitrary data that will be linked to a job and returned with results. Used to categorize jobs into batches, limited to 200 bytes [string]
--cache-inputs // Configures input file persistence. If enabled, input files may be cached on the compute node side for repeat computations. Should be disabled for dynamically changing data. Accepts strings "all" and "none", or a zero-indexed list of input files to cache (e.g. 0 2 would cache input files zero and two). [array] [default: "all"]
--checkpoint // Saves the state of running jobs and resumes them upon restart of the CLI. Prevents jobs with exactly identical parameters from running more than once, even after restarts of the CLI [boolean] [default: false]
--checkpointfile // Location and filename of the checkpoint file [string]
--commandline // Command line to execute. If using Docker images, command line should not include the command to execute Docker. It should be the command that will run inside the Docker container [string] [required]
--copies // Number of identical copies to execute [number] [default: 1]
--debug // Enables debug messages [boolean] [default: false]
--env // List of additional environment variables as key-value pairs to be passed to the job [array of string]
--eula // If running proprietary applications, marks whether end-user licence agreement of the application is accepted. Must be set to a string "accepted" for the jobs to be accepted into the system [string]
--exitafterstart // Exits the CLI after starting a job or checking job status. Useful to run multiple jobs in parallel without spiking up memory usage. Requires checkpointing to be enabled. After jobs are created, the CLI must be run again with the same parameters to retrieve results [boolean] [default: false]
--filechunksize // File part size to use when staging input files, in bytes [number] [default: 16384]
--hours // Maximum execution time allowed, in hours [number] [default: 1]
--inputfile // One or more input files as a local filename or an URL. If local files are specified, they will be staged to remote public URLs [array of string]
--instancetype // Sets the instance type to use for the job [string] [default: C.2x2]
--outputdir // Folder name to put output files to. If not specified, output files will be put to working dir. Wildcard %JOBKEY% will be replaced with the job ID [string]
--pollfrequency // RPC polling frequency in seconds [number] [default: 30]
--resubmit // Resubmits jobs if checkpointing is used and jobs are already completed [boolean] [default: false]
--result-storage // Location where output files should be stored. CurrentlyOptions supports the following options:
    * temporaryare "temporary" (free for up to two weeks)
, or   * "estuary" (Filecoin,; requires an Estuary API key to be specified in the --result-storage-config option)
 [string] [default: temporary]
--result-storage-config // Specifies any parameters that may be required when using non-default result storage destination [string]
--tag // Arbitrary data that will be linked to a job and returned with results. Limited to 1KB [string]
--useowndevices // Do not use the public network for job execution, but run on local devices instead. Useful for testing and debugging [default: false]
--help // Show help [boolean]

...

More info. See the overview of Input + Output Files for additional details on passing computing data in and out of Charity Engine.

...

If a lot of jobs use the same input file, or no input file at all, up to 500 jobs can be queued using a single CLI run by specifying the  --copies parameter:

Code Block
languagetextbash
ce-cli --app "docker:containername" --commandline "./run filename" --inputfile local_inputfile --copies 500 ...

...

Batches of jobs can be run using GNU Parallel (see Examples), or by using the --exitafterstart option to quickly launch all jobs, then running additional commands to retrieve the results. An example of the latter approach in a shell script is:

Code Block
languagetextbash
ce-cli --app "docker:containername" --commandline "./run filename1" --inputfile local_inputfile1 --checkpoint true --exitafterstart true ...

ce-cli --app "docker:containername" --commandline "./run filename2" --inputfile local_inputfile2 --checkpoint true --exitafterstart true ...

ce-cli --app "docker:containername" --commandline "./run filename3" --inputfile local_inputfile3 --checkpoint true --exitafterstart true ...

# Jobs have been created at this point, retrieve results

ce-cli --app "docker:containername" --commandline "./run filename1" --inputfile local_inputfile1 --checkpoint true ...

ce-cli --app "docker:containername" --commandline "./run filename2" --inputfile local_inputfile2 --checkpoint true ...

ce-cli --app "docker:containername" --commandline "./run filename3" --inputfile local_inputfile3 --checkpoint true ...

...

To write a simple “2+2” string to a text file, launch the Remote CLI as follows:

Code Block
languagetextbash
ce-cli --app “docker"docker:node”node" --commandline “echo"echo \”2"2+2\" > /local/output/out.txt”txt" --auth [KEY] --inputfile http://example.com/demo.txt local-file.txt

...

This simple concept can also be expanded to create 25 text files concurrently, one for each string in the series [ 1+1, 1+2, … 5+4, 5+5 ] by using GNU Parallel:

Code Block
languagetextbash
parallel -j 10000 ce-cli --app “docker"docker:node”node" --commandline “echo"echo \"{1}+{2}\" \> /local/output/out.txt”txt" --auth [KEY] ::: 1 2 3 4 5 ::: 1 2 3 4 5

...

Info

The useOwnDevices flag requires at least one of your devices to be running a Charity Engine client, attached with the same authorization key used to access the Remote CLI. For instructions on setting up the Charity Engine client, see Installing Charity Engine on Administered Resources.