Resource
BlogTopic
Edge DeliveryDate
Dec 11, 2019TL;DR: Using a Rust image manipulation library via WebAssembly, service workers can manipulate images on the fly. You can see a simple example of the code here and a fully-fledged example here.
Service workers can do several things on HTTP requests. Often they are used to generate HTML or proxy requests to different places based on runtime conditions. But they can also be used for other HTTP requests. In this article we’ll create a robust image manipulator inside the service worker.
While image manipulation libraries exist in pure JS, we are using this as an opportunity to leverage Rust and WebAssembly.
Specifically for this article, we will:
This article is not an in-depth walkthrough of Rust and other technologies. Rather, it is a brief walkthrough to reach our goal of image manipulation by starting with small steps so we can advance to the use of more advanced tooling.
Note: This tech demonstration uses Rust in StackPath’s Serverless Scripts. However, since image manipulation is primarily CPU bound, using Edge Compute instances would be a better performing, more robust approach. Any blogs using tools outside of StackPath may be impacted by breaking changes in a new version of the mentioned tool. Please utilize the version mentioned in this article. If you find that the current version does not work, please drop us a line and we’ll be glad to make the necessary edit.
Before we start advanced image manipulation, let’s first demonstrate calling a very basic Rust lib from basic JS.
Assuming we’re working in a folder called wasm-simple with Rust installed, create a crate called hello-rust:
cargo new --lib hello-rustThis creates a couple files in hello-rust. Navigate in there, and update the bottom portion of Cargo.toml to include some needed preferences:
[lib]
crate-type = ["cdylib"]
[lib]
crate-type = ["cdylib"]
[profile.release]
opt-level = 's'opt-level = ‘s’
The crate-type tells Rust that we want to compile this as a C dynamic library which means the externs are available via C FFI conventions which LLVM’s compiler exports to WASM. The opt-level tells LLVM to optimize for code size on release builds.
Update the src/lib.rs with a simple call accepting and returning primitives:
#[no_mangle]
pub extern fn add_numbers(x: u32, y: u32) -> u32 {
x + y
}Now we can compile for WASM:
cargo build --target wasm32-unknown-unknownThis target may have to be installed if not already present (e.g. rustup target add wasm32-unknown-unknown if you’re using rustup). The build command builds a WASM file in debug mode at
target/wasm32-unknown-unknown/debug/hello_rust.wasm (~1.4MB as of this writing).
To build in release mode:
cargo build --target wasm32-unknown-unknown --releaseThis is in the release folder instead of the debug folder but the size doesn’t shrink much. We’ll get to that later, but for now we’ll just use the debug version in our script.
Now that we have a WASM file, we’ll create a TypeScript project at the parent directory (i.e. wasm-simple). In that directory, create a simple package.json for the project as private since we aren’t publishing it:
{
"private": true
}We want to use the cloudworker utility to run our service worker locally, so install it and save it as a dev dependency in package.json:
npm install --save-dev @dollarshaveclub/cloudworkerAnd update the package.json with a shortcut to run it with src/index.js:
{
"private": true,
"scripts": {
"cloudworker": "cloudworker -r src/index.js"
},
"devDependencies": {
"@dollarshaveclub/cloudworker": "^0.1.1"
}
}Now let’s create our simple script in src/index.js:
const wasmBuf = new Uint8Array([/*TODO*/])
const wasmModule = new WebAssembly.Module(wasmBuf)
const wasmInstance = new WebAssembly.Instance(wasmModule)
addEventListener('fetch', event => event.respondWith(handleRequest(event.request)))
async function handleRequest(request) {
try {
return new Response('5 + 6 = ' + wasmInstance.exports.add_numbers(5, 6))
} catch (e) {
return new Response('Error: ' + (e.stack || e), { status: 500 })
}
}This will take the byte array from wasmBuf and create an instance that we can call. Unlike browsers and Nodejs scripts, we don’t load our WASM file from an external source. It needs to be embedded, hence the TODO above.
There are several ways we can embed the WASM file, but as a simple approach for now, let’s just have a script embed it.
First, add a generated-code comment warning above the wasmBuf var so we can find it:
// Code generated on build, do not edit this line or the next manually.
const wasmBuf = new Uint8Array([/*TODO*/])Then, adjacent to the src directory, create the code generator at code_gen.js:
const fs = require('fs')
// Load WASM
const wasm = fs.readFileSync('hello-rust/target/wasm32-unknown-unknown/debug/hello_rust.wasm')
// Load JS file
let js = fs.readFileSync('src/index.js', 'utf8')
// Replace the Uint8Array with a new one
js = js.replace(/// Code generated on build.*r?n.*r?n/,
"// Code generated on build, do not edit this line or the next manually.n" +
"const wasmBuf = new Uint8Array([" + new Uint8Array(wasm) + "])n")
fs.writeFileSync('src/index.js', js)This is a simple Nodejs script that reads the file and replaces the line after the code-gen comment with the embedded byte array. Since we may want to run this and the cargo build repeatedly, let’s add a script to package.json:
{
"private": true,
"scripts": {
"build-wasm": "cd hello-rust && cargo build --target wasm32-unknown-unknown && cd .. && node code_gen.js",
"cloudworker": "cloudworker -r src/index.js"
},
"devDependencies": {
"@dollarshaveclub/cloudworker": "^0.1.1"
}
}Now run it:
npm run build-wasmIf successful, our JS file just ballooned to over 4MB! Some of the size problems will be solved with later tooling. For now, let’s just confirm that our script works.
Run:
npm run cloudworkerNavigating to http://localhost:3000/ shows:
5 + 6 = 11Now it should be clear how the underlying WASM system works. With this understanding, we can use high level tooling to make this better…
Using webpack with wasm-pack will not only take advantage of other Rust WASM tooling to shrink our output, but it will generate TypeScript declarations that can help us during development.
Instead of retrofitting the wasm-simple project, we’ll create a new one in wasm-better. Like before, in wasm-better, create a hello-rust lib:
cargo new --lib hello-rustThen, after the top metadata in hello-rust/Cargo.toml, add:
[lib]
crate-type = ["cdylib"]
[dependencies]
wasm-bindgen = "0.2"
console_error_panic_hook = "0.1"
[dependencies.web-sys]
version = "0.3"
features = [
"console"
]
[profile.release]
lto = true
opt-level = "s"We did a few extra things here and added three new dependencies.
wasm-bindgen is a library for making it easier to bridge Rust and JS with exports and imports.console_error_panic_hook is a library that sets a panic hook to log to console.error.web-sys is all of the common JS libraries generated for use in Rust. With web-sys, each JS API feature has to be specified to be included. Here we included console so we can create console logs.lto option to release builds to have LLVM perform link-time optimizations.We’ll make a bit more complex script in hello-rust/src/lib.rs than we did in the last section:
use wasm_bindgen::prelude::*;
// Macro for console logs
macro_rules! console_log {
($($t:tt)*) => (web_sys::console::log_1(&format!($($t)*).into()))
}
#[wasm_bindgen]
pub fn init() {
console_error_panic_hook::set_once();
}
#[wasm_bindgen]
pub fn run(url: &str) -> String {
console_log!("Caller came from URL: {}", url);
format!("You came from URL: {}", url)
}This has a console_log! macro, an exported init function to install the panic hook, and an exported run function that logs to console and returns a string.
The #[wasm_bindgen] attribute marks the function as exported to WASM. We use strings here since their allocation from JS is more complicated than the numeric primitives in our last example.
Previously we built with Cargo directly, but now we will build with wasm-pack. However, instead of using wasm-pack directly we will use the wasm-pack plugin inside of webpack. But we do need wasm-pack installed, so run this anywhere:
cargo install wasm-packTo create the service worker project, start with a simple package.json at the root of the wasm-better directory.
{
"private": true
}Install TypeScript, webpack, and the TypeScript loader:
npm install --save-dev typescript webpack webpack-cli ts-loaderAlso install the cloudworker and wasm-pack plugin:
npm install --save-dev @dollarshaveclub/cloudworker @wasm-tool/wasm-pack-pluginSince we are now using webpack to bundle our TypeScript and WASM into a single file, we add this scripts section to the package.json with the build commands and the cloudworker runner:
"scripts": {
"dev": "webpack -w --mode=development",
"build": "webpack --mode=production",
"cloudworker": "cloudworker -r dist/index.js"
},To configure webpack to compile our TypeScript and WASM, we add the following webpack.config.js file at the project root:
const path = require('path')
const webpack = require('webpack')
const WasmPackPlugin = require('@wasm-tool/wasm-pack-plugin')
module.exports = (env, argv) => {
return {
devtool: '',
entry: './src/index.ts',
output: {
filename: 'index.js',
// Needed to not use "window" as global
globalObject: 'this'
},
resolve: {
extensions: ['.js', '.ts', '.wasm']
},
module: {
rules: [
// Custom WASM loader
{
test: /.wasm$/,
type: 'javascript/auto',
loader: path.resolve(__dirname, './wasm_loader.js')
},
{ test: /.ts$/, loader: 'ts-loader' }
]
},
plugins: [
new WasmPackPlugin({
crateDirectory: path.resolve(__dirname, './hello-rust'),
forceMode: argv.mode
}),
// The script gets so large webpack starts chunking it which we don't want
new webpack.optimize.LimitChunkCountPlugin({ maxChunks: 1 })
]
}
}There are a few things happening here beyond the standard webpack+TypeScript configuration:
globalObject to this. By default webpack uses window as the the global variable name which is not the global variable name in service workers..wasm files using our custom wasm_loader.js. webpack’s default WASM loader does not work well with our need to have it embedded and loaders like arraybuffer-loader do not instantiate WASM properly, so we’ll write our own (below).WasmPackPlugin, reference our Rust lib, and have it use the mode from the CLI arguments.LimitChunkCountPlugin to make sure webpack only generates one file. By default webpack chunks files that get too large which is not what we want for a single-script service worker.Also, since we are using TypeScript, we need a tsconfig.json file at the root:
{
"compilerOptions": {
"target": "esNext",
"module": "esNext",
"strict": true,
"esModuleInterop": true,
"lib": ["es2015", "webworker"],
"sourceMap": true
},
"include": [
"./src/**/*"
]
}Most of this is standard for using service workers with TypeScript. Also, before writing our actual TypeScript, we need the aforementioned custom WASM loader that embeds the WASM file and generates code to instantiate the WASM instance with imports.
Place this at wasm_loader.js at the project root:
module.exports = function(source) {
// We'll just do simple instantiation in this loader
let code = "// Load WASMn"
const wasmBuf = new Uint8Array(source)
code += "const wasmMod = new WebAssembly.Module(new Uint8Array([" + wasmBuf + "]))n"
// Load up the module to get import metadata and setup imports. We'll first
// group by module name, then create a bunch of function proxies. This is a
// hacky approach around proper import resolution.
const wasmMod = new WebAssembly.Module(wasmBuf)
// Key is module, val is array of imports
const importMap = {}
WebAssembly.Module.imports(wasmMod).forEach(mport => {
if (mport.kind !== 'function') throw new Error('Only function imports supported')
if (!importMap[mport.module]) importMap[mport.module] = []
importMap[mport.module].push(mport)
})
let importCode = ''
for (let modName in importMap) {
let parts = ''
importMap[modName].forEach(mport => {
if (parts) parts += ','
parts += "n '" + mport.name + "': function() { return require('" +
modName + "')." + mport.name + ".apply(null, arguments) }"
})
if (importCode) importCode += ','
importCode += "n '" + modName + "': {" + parts + "n }"
}
code += "const wasmImports = {" + importCode + "n}n"
code += "const wasmInst = new WebAssembly.Instance(wasmMod, wasmImports)n"
code += "module.exports = wasmInst.exportsn"
return code
}
// We need it as a buffer instead of a string
module.exports.raw = trueThe details of the script are beyond the scope of this article, but it takes a very primitive approach
around JS import resolution just for our limited needs. As other loaders mature, it would be best to use them or change this one to be more robust if there are more complex import needs.
Let’s create an empty file index.ts at src/index.ts and compile the Rust crate before we write our TypeScript so we have declarations to work with:
npm run buildIt will take a bit to build the crate the first time. You might get an error from webpack saying that there is no entry point which is OK because we haven’t written it yet.
wasm-pack did all of the hard work for us of compiling the crate and placing the artifacts at hello-rust/pkg. And, in addition to the JS and TypeScript declarations in there, a nice and small ~40KB WASM file was built after all optimizations.
Finally, after all of that we can write our TypeScript code. At src/index.ts:
import * as helloRust from '../hello-rust/pkg/index'
helloRust.init()
addEventListener('fetch', ((event: FetchEvent) => {
event.respondWith(handleRequest(event.request))
}) as EventListener)
async function handleRequest(request: Request) {
try {
return new Response('Hello. ' + helloRust.run(request.url))
} catch (e) {
return new Response('Error: ' + (e.stack || e), { status: 500 })
}
}Now we can run it. We’ll run it in debug mode instead of production mode while we work. In one terminal:
npm run devThis may take a bit on the first compile. After compile, this stays open and will recompile if we change our Rust or TypeScript code.
Now, in another terminal, we can run the cloudworker listening for updates:
npm run cloudworkerNavigating to http://localhost:3000/ shows:
Hello. You came from URL: http://localhost:3000/Feel free to edit the Rust or TypeScript code to watch it reload.
Running npm run build instead of dev will create an optimized all-in-one script at dist/index.js that can be uploaded/pasted as a Serverless Script.
Now that we’ve learned how to do more complex calls into Rust we can move on to our original goal of image manipulation.
We have an approach now to call complex Rust code from TypeScript and generate a single bundle. So let’s create some code to fetch an image and resize it to a custom size.
Copy everything we’ve done before to a new directory called wasm-image and change the Rust lib/dir name to rust-image and all references to it. Confirm it works with npm run build and npm run cloudworker.
First, let’s just read an image with the image-rs library and write it back out.
We’ll do the fetching inside Rust, so we need some more web-sys APIs and the js-sys library for other JS values. We also need the image-rs library but we only want PNG and JPEG support for now.
Finally, since we’ll be using JS promises as Rust futures, we want futures and wasm-bindgen-futures. The dependencies area in rust-image/Cargo.toml should read:
[dependencies]
futures = "0.1"
wasm-bindgen = "0.2"
wasm-bindgen-futures = "0.3"
js-sys = "0.3"
console_error_panic_hook = "0.1"
[dependencies.web-sys]
version = "0.3"
features = [
"console",
"Headers",
"Response",
"ResponseInit",
"WorkerGlobalScope",
]
[dependencies.image]
version = "0.22"
default-features = false
features = [
"jpeg",
"png_codec",
]
use futures::Future;
use std::io::Cursor;
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsCast;
use wasm_bindgen_futures::{future_to_promise,JsFuture};
#[allow(unused_macros)]
macro_rules! console_log {
($($t:tt)*) => (web_sys::console::log_1(&format!($($t)*).into()))
}
#[wasm_bindgen]
pub fn init() {
console_error_panic_hook::set_once();
}
#[wasm_bindgen]
pub struct WorkingImage {
fut: Box>
}
struct ImageInfo {
image: image::DynamicImage,
headers: web_sys::Headers,
format: image::ImageFormat,
}
#[wasm_bindgen]
impl WorkingImage {
#[wasm_bindgen(constructor)]
pub fn new(url: &str) -> WorkingImage {
// Start fetch
let global = js_sys::global().unchecked_into::();
let resp_promise = global.fetch_with_str(url);
let fut = Box::new(JsFuture::from(resp_promise)
// Read buffer if response is OK
.and_then(|resp_val| {
let resp: web_sys::Response = resp_val.dyn_into().unwrap();
if !resp.ok() {
// Just throw the response on failure
return Err(JsValue::from(resp));
}
Ok((resp.headers(), resp.array_buffer().unwrap()))
})
// Extract buffer from promise
.and_then(move |(headers, buf_promise)| {
JsFuture::from(buf_promise).map(move |buf_val| { (headers, buf_val) })
})
// Copy buffer to byte array and load image
.and_then(move |(headers, buf)| {
let uint8_arr = js_sys::Uint8Array::new(&buf);
let mut bytes = vec![0; uint8_arr.length() as usize];
uint8_arr.copy_to(&mut bytes);
let format = image::guess_format(&bytes).map_err(err_img_to_js)?;
Ok(ImageInfo {
image: image::load_from_memory_with_format(&bytes, format).map_err(err_img_to_js)?,
headers: headers,
format: format,
})
}));
WorkingImage { fut: fut }
}
pub fn build(self) -> js_sys::Promise {
future_to_promise(
self.fut.and_then(move |info| {
// We'll use the same headers, but remove length
let headers = web_sys::Headers::new_with_headers(&info.headers).unwrap();
headers.delete("Content-Length").unwrap();
// Write to a buffer
let mut buf = Cursor::new(Vec::new());
info.image.write_to(&mut buf, info.format).map_err(err_img_to_js)?;
Ok((headers, buf))
})
.and_then(|(headers, buf)| {
// Build the response
let body = js_sys::Uint8Array::from(buf.get_ref().as_slice());
let resp = web_sys::Response::new_with_opt_buffer_source_and_init(
Some(&body), web_sys::ResponseInit::new().headers(&headers))?;
Ok(JsValue::from(resp))
})
)
}
}
fn err_img_to_js(i: image::ImageError) -> JsValue {
JsValue::from(js_sys::Error::new(&format!("Image error: {}", i)))
}
The code creates a struct that is really just a wrapper around a boxed future that contains image info. The exported constructor uses fetch to obtain the image and then loads it into a DynamicImage in the image library. Then there is a build method that returns a Promise of the newly generated image using the same format it came in as.
Here is the TypeScript code at src/index.ts:
import * as rustImage from '../rust-image/pkg/index'
rustImage.init()
addEventListener('fetch', ((event: FetchEvent) => {
event.respondWith(handleRequest(event.request))
}) as EventListener)
async function handleRequest(request: Request) {
let image: rustImage.WorkingImage|null = null
try {
// Pull the URL param
const requestParams = new URL(request.url).searchParams
const imageUrl = requestParams.get('url')
if (!imageUrl) return new Response('Missing URL param', { status: 400 })
// Load it
image = new rustImage.WorkingImage(imageUrl)
// Build it
return await (image.build() as Promise)
} catch (e) {
if (e instanceof Response) return e
return new Response('Error: ' + e.stack || e, { status: 500 })
} finally {
if (image) try { (image as any).free() } catch (e) { }
}
} Besides some workaround of TypeScript typings, it is pretty straightforward. Now, running npm run build and starting
npm run cloudworker, navigating to http://localhost:3000/?url=https://picsum.photos/id/1003/500/500 displays the
image.
Now add a resize method to Rust in the impl at rust-image/src/index.ts line 18:
pub fn resize(self, w: u32, h: u32, exact: bool) -> WorkingImage {
WorkingImage {
fut: Box::new(self.fut.map(move |info| {
ImageInfo {
image: if exact {
info.image.resize_exact(w, h, image::FilterType::Lanczos3)
} else {
info.image.resize(w, h, image::FilterType::Lanczos3)
},
..info
}
})),
}
}Update TypeScript to support it:
// Resize if params present
const w = requestParams.get('w')
if (w) {
const wInt = parseInt(w)
const hInt = parseInt(requestParams.get('h') || w)
if (Number.isNaN(wInt) || Number.isNaN(hInt)) {
return new Response('Invalid width or height', { status: 400 })
}
image = image.resize(wInt, hInt, requestParams.get('exact') !== null)
}Now try adding &w=64 to the URL from before. Or &w=64&h=150&exact. The resulting dist/index.js is less than 1MB uncompressed and ~170KB with default gzip compression.
Note: This is just an example and, for security and bandwidth saving reasons, developers would be wise to limit the URLs they accept (or hardcode to a local server path or something).
In this article we showed you how to do simple image manipulation via Rust. While our example was a generic resizing operation, it can be used to do any number of operations tailored to specific needs.
Some possibilities include:
Based on this article, a version of this image manipulation script was created with many more features and is available here.