The Charity Engine Smart Proxy enables running Javascript applications within a full-featured web browser on a vast network of volunteer computing devices.
Smart Proxy service currently uses PhantomJS to run a headless browser and exposes its API; we may add additional browsers/APIs in the future.
Smart proxy service is under rapid development
The features may get added, changed or removed at any time without advance notice. The service is in alpha state.
This document is intended to obtain feedback on the roadmap for implementation and should not be considered final documentation.
Unless otherwise noted, the mechanisms of the Charity Engine Distributed Proxy for authenticating to the proxy and proxy configuration still apply as Smart Proxy is an extension of the generic Distributed Proxy. Refer to Charity Engine Distributed Proxy documentation for details.
Initiating and configuring Smart Proxy crawls
Smart Proxy crawls are initiated by connecting to the Distributed Proxy service and supplying additional HTTP headers.
x-proxy-phantomjs-script-url
A URL for the script to run on the URL that was supplied to the proxy.
While currently any URL is accepted, the service will only allow whitelisted URLs or hostnames in the future.
x-proxy-phantomjs-script-md5
An MD5 hash of the script file defined within x-proxy-phantomjs-script-url HTTP header. Used to verify that the scripts get downloaded correctly. Scripts that do not match the supplied MD5 hash will no be run.