PNG Proxy: returns a screenshot of requested website

For many years setting up environment for Selenium tests was a tricky task for DevOps engineers. With appearence of Docker it became a bit easier, but still, you had to run a couple containers and keep them alive.

Since 2017 @smithclay made possible to run Webdriver tests inside AWS Lambda, but not so many people realized what creative possibilities it brings. 

You make a request to proxy and it returns you a screenshot of target website

For example, with small modifications we can make a PNG Proxy: you request proxy by http, it triggers Lambda function, runs test inside Lambda and returns screenshot in png format back to you. Possible use cases:

• See how website looks with different screen resolutions.
• On mobile see a website which does not support small screen resolutions.
• Open website on smart TV without need to log in (Selenium can submit login form for you).

How it works?

Source code of PNG proxy could be found on my GitHub maxmode/chromecrawler_terraform.  The architecture of solution consists of AWS API Gateway, which triggers AWS Lambda function. Inside Lambda function there is a headless chrome browser, chromdriver to control it, and a small handler script on Node.js, which allows to execute Selenium Webdriver tests on Node.js.

I made this project, being inspired by work of @smithclay. Two things were added to original repository:

• Deployment with terraform. Terraform creates Lambda package (zip archive) and deploys it together with API Gateway and required permissions.
• Passing callback variable into the test, so that test decides what API Gateway will return. Which status code, which body, which headers. By calling a callback it is even possible to return a PNG image in base64 encoded format.

Example of test, which would open website moskvychev.com and return screenshot of it:

// This uses the selenium-webdriver 3.4 package.
// $browser = webdriver session
// $driver = driver libraries
// console.log will output to AWS Lambda logs (via Cloudwatch)
// callback is a callback from Lambda's index.handler.

function takeScreenshot(statusCode = 200) {
  $browser.getTitle()
    .then(function(title) {
    console.log("Screenshot will be made for page with title: " + title);
    $browser.takeScreenshot()
      .then(
      function(img, err) {
        callback(null, {
          statusCode: statusCode,
          headers: {
            'Content-Type': 'image/png',
            'Cache-Control': "public, max-age=30"
          },
          body: img,
          isBase64Encoded: true
        });
        console.log('---');
        console.log('Image size: ' + Math.round(img.length/1024) + 'Kb');
        console.log('---');
        console.log('Status code: ' + statusCode);
      }
    )
  });
}

let url = 'https://moskvychev.com';
console.log('About to open "' + url + '"');
$browser.manage().window()
  .setSize(1280, 500);
$browser.get(url);
$browser.sleep(1000)
  .then(function(){
  takeScreenshot(200);
});

And result:

Further thoughts

AWS Lambda could totally change a way we are running our Selenium tests. No need any more to create a server/container with headless browsers and webdriver environment. Just write your test and use Lambda to execute it.

Also integration with API Gateway creates some interesting workflows. Only your imagination is a limit what you can do with it, starting with "PNG proxy" and ending with highly scalable crawling systems.