Custom functions

Cloudinary supports injecting a custom function into the image transformation pipeline. You can either use a remote function/lambda as your source, or run WebAssembly functions from a compiled wasm file uploaded to your account.

To specify a custom function to call, use the custom_function parameter (fn in URLs). The parameter accepts an object detailing the function to inject as follows:

Parameter Description
function_type The type of function to run, either 'remote' or 'wasm'
source The source of the custom function, either the public_id of the wasm file or the URL of the remote function

Tip
Once requested, derived images from custom functions are cached like any other derived images generated by Cloudinary: modifying your custom function will not generate a new image for the same dynamic URL as the URL itself has not changed. If you need to bypass this issue when you change a custom function, we recommend adding a version component to the Cloudinary URL.

WebAssembly functions

Compiled wasm files may be uploaded as raw authenticated resources to your Cloudinary account and then referenced in a custom function. Use the custom_function parameter with the function_type set to "wasm" (fn_wasm in URLs), and the source parameter set to the public_id of your compiled wasm file.

For example, to deliver the 'sample' image after running the WebAssembly functions located in a compiled wasm file:

  1. Upload your compiled wasm file to your account as an authenticated raw file:

    Ruby:
    Cloudinary::Uploader.upload("my_example.wasm", 
      :use_filename => true,
      :type => :authenticated,
      :resource_type => :raw)
    PHP:
    \Cloudinary\Uploader::upload("my_example.wasm", 
      array(
        "use_filename" => TRUE,
        "type" => "authenticated",
        "resource_type" => "raw"));
    Python:
    cloudinary.uploader.upload("my_example.wasm", 
      use_filename = true,
      type = "authenticated",
      resource_type = "raw")
    Node.js:
    cloudinary.v2.uploader.upload("my_example.wasm", 
      { use_filename: true,
        type: "authenticated",
        resource_type: "raw" }, 
      function(error, result) {console.log(result, error); });
    Java:
    cloudinary.uploader().upload("my_example.wasm", 
      ObjectUtils.asMap(
        "use_filename", "true",
        "type", "authenticated",
        "resource_type", "raw"));
    .Net:
    var uploadParams = new RawUploadParams(){  // by default, ResourceType is already set to "raw"
      UseFilename = true,
      Type = "authenticated",
      File = new FileDescription(@"my_example.wasm")};
    var uploadResult = cloudinary.Upload(uploadParams);
  2. Deliver the sample image after running the WebAssembly functions located in your now uploaded my_example.wasm file:

    Ruby:
    cl_image_tag("sample.jpg",
      :custom_function => {
        :function_type => "wasm", 
        :source => "my_example.wasm"})
    PHP:
    cl_image_tag("sample.jpg", array(
      "custom_function" => array(
        "function_type" => "wasm", 
        "source" => "my_example.wasm")))
    Python:
    CloudinaryImage("sample.jpg").image(
      custom_function = {
        "function_type" = "wasm", 
        "source" = "my_example.wasm"})
    Node.js:
    cloudinary.image("sample.jpg", {
      custom_function:{
        function_type: "wasm", 
        source: "my_example.wasm"}})
    Java:
    cloudinary.url().transformation(new Transformation()
      .customFunction(wasm("my_example.wasm")))
      .imageTag("sample.jpg");
    JS:
    cloudinary.imageTag('sample.jpg', {
      customFunction: new cloudinary.CustomFunction()
        .functionType("wasm")
        .source("my_example.wasm")}).toHtml();
    jQuery:
    $.cloudinary.image('sample.jpg', {
      customFunction: new cloudinary.CustomFunction()
        .functionType("wasm")
        .source("my_example.wasm")})
    React:
    <Image publicId="sample.jpg">
      <Transformation customFunction={{
        functionType: "wasm",
        source: "my_example.wasm"}} />
    </Image>
    Angular:
    <cl-image public-id="sample.jpg">
      <cl-transformation customFunction={
       functionType: "wasm",
        source: "my_example.wasm"}>
      </cl-transformation>
    </cl-image>
    .Net:
    cloudinary.Api.UrlImgUp.Transform(new Transformation()
      .CustomFunction(CustomFunction.Wasm("my_example.wasm")))
      .BuildImageTag("sample.jpg")
    Android:
    MediaManager.get().url().transformation(new Transformation()
      .customFunction(new CustomFunction()
        .functionType("wasm")
        .source("my_example.wasm"))
      .generate("sample.jpg");
    iOS:
    imageView.cldSetImage(cloudinary.createUrl()
      .setTransformation(CLDTransformation()
        .setCustomFunction(CLDCustomFunction()
          .setFunctionType("wasm")
          .setSource("my_example.wasm")))
      .generate("sample.jpg")!, cloudinary: cloudinary)

    The code generates a URL similar to:

    https://res.cloudinary.com/demo/image/upload/fn_wasm:my_example.wasm/sample.jpg

WebAssembly contract

Note
The samples below are given in the Rust programming language which is then compiled to a wasm target - but this can be achieved with any language that compiles to .wasm.

The WebAssembly functions in your compiled wasm file need to comply with a specific interface - you must provide the following 3 public functions:

1. alloc

Your alloc method needs to allocate memory according to the size given and then return a pointer to the allocated memory.

    alloc(size: usize) -> *mut u8

2. dealloc

Your dealloc method should deallocate memory at the given pointer, according to the size given.

    dealloc(ptr: *mut u8, cap: usize)

3. transform

Your transform method should include the code you want to perform the actual transformation of the image. The method receives the image width, image height, a pointer to the pixel buffer (of size = width x height x 4) where the pixel scheme is guaranteed to be RGBA interleaved, a pointer to the metadata, and the metadata size.

The metadata is given as a JSON structure which contains the following information if defined for the image: "context" as a map of string to string, "tags" as an array of strings, and "variables" as a map from string (variable name) to any value.

The transform function must return a pointer to the output buffer that contains the following information: width as a 32 bit BigEndian, followed by the height as a 32 bit BigEndian, followed by the image pixel buffer (RGBA scheme again).

    transform(width: u32, height: u32, image_ptr: *mut u8, meta_ptr: *mut u8, meta_size: usize) -> u32

WebAssembly example

The following example applies a blur effect to an image:

#![feature(exact_chunks)]

#[macro_use]
extern crate serde_derive;
extern crate image;
extern crate serde;
extern crate serde_json;
extern crate byteorder;

use std::mem;
use byteorder::{ WriteBytesExt, BigEndian};
use image::{RgbaImage, imageops};

#[derive(Deserialize, Debug)]
struct Metadata {
    context: Option<std::collections::HashMap<String, String>>,
    tags: Option<Vec<String>>,
    variables: Option<std::collections::HashMap<String, i32>>,
}

#[no_mangle]
pub extern "C" fn alloc(size: usize) -> *mut u8 {
    let mut buf = Vec::<u8>::with_capacity(size);
    let ptr = buf.as_mut_ptr();
    mem::forget(buf);
    return ptr;
}

#[no_mangle]
pub extern "C" fn dealloc(ptr: *mut u8, cap: usize) {
    unsafe  {
      let _buf = Vec::from_raw_parts(ptr, 0, cap);
    }
}

#[no_mangle]
pub extern "C" fn transform(width: u32, height: u32, image_ptr: *mut u8, meta_ptr: *mut u8, meta_size: usize) -> u32 {
  let size = (width * height * 4) as usize;
  let bytes = unsafe {Vec::from_raw_parts(image_ptr, size, size)};
  let meta_bytes = unsafe {Vec::from_raw_parts(meta_ptr, meta_size, meta_size)};
  let metadata: Metadata = serde_json::from_slice(&meta_bytes).expect("Failed to deserialize metadata json");
  host_trace(format!("{:?}", metadata));
  let (out_w, out_h, mut out_buffer) = blur(width, height, bytes, metadata);
  let mut dims = vec![];
  let _ = dims.write_u32::<BigEndian>(out_w);
  let _ = dims.write_u32::<BigEndian>(out_h);
  dims.append(&mut out_buffer);
  let out_buffer = dims;
  let out_ptr = out_buffer.as_ptr() as u32;
  mem::forget(out_buffer);
  out_ptr  
}
fn host_trace(x: String) {
  let buf = x.into_bytes();
  let length = buf.len();
  let ptr = buf.as_ptr();
  unsafe { trace(ptr as u32, length as u32) }
}
extern "C" {
  pub fn trace(x: u32, length: u32);
}
fn blur(width: u32, height: u32, bytes: Vec<u8>, _metadata: Metadata) -> (u32, u32, Vec<u8>) {
  let ref img = RgbaImage::from_raw(width, height, bytes).unwrap();
  let subimg = imageops::blur(img, 2.5);
  let out_w = subimg.width();
  let out_h = subimg.height();
  let out_buffer = subimg.into_raw();
  (out_w, out_h, out_buffer)
}

Remote functions

You can call a remote function/lambda as part of the transformation chain. The remote function receives an input image file (PNG) along with metadata, and must return an image file (optionally along with metadata also). Use the custom_function parameter with the function_type set to "remote" (fn_remote in URLs), and the source parameter set to the URL of the custom function. The delivery URL also needs to be signed, which means also adding the sign_url parameter set to "true" to the SDK method.

For example, to deliver the 'sample' image after running the remote function located at 'https://my.example.custom/function':

Ruby:
cl_image_tag("sample.jpg", 
  :sign_url => true,
  :custom_function => {
    :function_type => "remote", 
    :source => "https://my.example.custom/function"})
PHP:
cl_image_tag("sample.jpg", array(
  "sign_url" => true,
  "custom_function" => array(
    "function_type" => "remote", 
    "source" => "https://my.example.custom/function")))
Python:
CloudinaryImage("sample.jpg").image(
  sign_url = True,
  custom_function = {
    "function_type" = "remote", 
    "source" = "https://my.example.custom/function"})
Node.js:
cloudinary.image("sample.jpg", {
  sign_url: true,
  custom_function:{
    function_type: "remote", 
    source: "https://my.example.custom/function"}})
Java:
cloudinary.url().transformation(new Transformation()
    .signed(true)
    .customFunction(remote("https://my.example.custom/function")))
  .imageTag("sample.jpg");
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
    .Signed(true)
    .CustomFunction(CustomFunction.Remote("https://my.example.custom/function")))
  .BuildImageTag("sample.jpg")

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/s--89bc7b34--/fn_remote:aHR0cHM6Ly9teS5leGFtcGxlLmN1c3RvbS9mdW5jdGlvbg==/sample.jpg

Note
The Cloudinary SDKs automatically generate a base 64 encoded URL from the source parameter: you will need to supply the source URL in base 64 with padding if you generate the delivery URL in your own code.

Request structure

The URL of your remote function receives the following information in the HTTP POST request from Cloudinary:

Request headers:

  • X-Cld-Timestamp - an integer value representing the time the request was sent in Unix time.
  • X-Cld-Signature - a signature string for verification: the string is an SHA-1 Hex-Digest of the request body + timestamp + your API secret.

Request body:

  • file - the (binary) image file.
  • metadata - a JSON structure that includes the current_page, tags, coordinates (each key represents a coordinate source) and variables (key-value pairs representing variable names and values). For example:

    {
      "current_page": 1,
      "tags": [],
      "coordinates": {
        "eyedea": {
          "coords": [],
          "kind": "eyedea",
          "on_original": true,
          "failed": false
        }
      },
      "variables": {
        "z": 5,
        "foo": 10
      }
    }

Response structure

Your response should include the image Content-Type in the header and the image data in the body. For example, if using API Gateway to host your remote function then the response would be similar to:

{
  "statusCode": 200,
  "headers": {
    "Content-Type": "image/jpeg",
    "Content-Length": "result.length"
  },
  "isBase64Encoded": true,             // base64 is restriction imposed by API Gateway
  "body": "result.toString('base64')"   // image data encoded as base64 with padding
}

The body of your response can contain just the image data or you can include new metadata as follows:

  1. Start the response body with the string 'CLDB'.
  2. Add 4 bytes (BigEndian) describing the byte length of the result image buffer.
  3. Add the image buffer.
  4. Add 4 bytes (BigEndian) describing the byte length of the result metadata JSON buffer.
  5. Add the metadata buffer.
CLDB + length_of_image_buffer + image_buffer + length_of_metadata buffer + metadata_buffer

Sample remote function

The following code is an example of a remote function that is hosted on AWS lambda behind an API gateway and is written in JavaScript. The function resizes an image to a width of 314px, adds a text overlay of the current time, and also returns new metadata:

const im = require('imagemagick');
const multipart = require('aws-lambda-multipart-parser');
const fs = require('fs');
const fail = (message) => {
    console.log(message);
    throw new Error(message);
};
const perform = (operation, args) => new Promise((resolve, reject) => im[operation](args, (err, res) => {
    if (err) {
        console.log(`${operation} operation failed:`, err);
        reject(err);
    } else {
        console.log(`${operation} completed successfully`);
        resolve(res);
    }
}));
const postProcessResource = (resource, fn) => {
    let ret = null;
    if (resource) {
        if (fn) {
            ret = fn(resource);
        }
        try {
            fs.unlinkSync(resource);
        } catch (err) {
            // Ignore
        }
    }
    return ret;
};
const transform = async (file) => {
    // current time as string
    const date = (new Date()).toDateString();
    // transformation in imagemagick: resize to 314px, overlay text at x=5px, y=20px.
    const customArgs = ['-resize', '314x', '-fill', 'blue', '-draw', `text 5,20 '${date}'`];
    // prepare input and output files
    let inputFile = null;
    let outputFile = null;
    inputFile = `/tmp/inputFile`;
    fs.writeFileSync(inputFile, file);
    customArgs.unshift(inputFile);
    outputFile = `/tmp/outputFile.jpg`;
    customArgs.push(outputFile);
    // actual conversion
    const output = await perform('convert', customArgs);
    postProcessResource(inputFile);
    if (outputFile) {
        return postProcessResource(outputFile, (file) => new Buffer(fs.readFileSync(file)));
    }
    // Return the command line output as a debugging aid
    return output;
};
exports.handler = async (event, context, callback) => {
    const parsedRequest = multipart.parse(event, false);
    const file = parsedRequest.file.content;
    return transform(file).then((result) => {
      // return the image and new metadata.   
      if (event.queryStringParameters && event.queryStringParameters.cldb) {
          const bodyLengthBuf = new Buffer(4);
          const bodyLength = result.length;
          bodyLengthBuf.writeUInt32BE(bodyLength);
          const metadata = Buffer.from(JSON.stringify({"coordinates": {"custom": [[45,57,100,120]]}}));
          const metadataLengthBuf = new Buffer(4);
          const metadataLength = metadata.length;
          metadataLengthBuf.writeUInt32BE(metadataLength);
          result = Buffer.concat([Buffer.from('CLDB'), bodyLengthBuf, result, metadataLengthBuf, metadata], 3*4 + metadataLength + bodyLength);
      }
      callback(null, {
        statusCode: 200,
        headers: { 'Content-Type': 'image/jpeg', 'Content-Length': result.length},
        isBase64Encoded: true,
        body: result.toString('base64')
      });
    }).catch((error) => {
      callback(null, {
        statusCode: 502,
        headers:    { 'Content-Type': 'application/json' },
        body:    `{"error": "Error manipulating image ${error}"}`
      });
    });
};

Preprocessing custom functions

You can insert your custom function earlier in the transformation chain, before Cloudinary does any processing whatsoever on the image. Whereas the remote function option described above is sent an image in PNG format, a preprocessing remote function is sent the original image file, as uploaded to Cloudinary. For example, you can upload images in a format Cloudinary does not support for transformations and use a custom function to return an image format that Cloudinary does support.

Only Remote Functions are supported for preprocessing as described above, except for the following differences:

  1. Use the custom_pre_function parameter (fn_pre in URLs) to call the custom function to preprocess. The parameter accepts the same type of object as the custom_function parameter, detailing the function_type ('remote') and source.
  2. The preprocessing function is sent the original unaltered image, plus any defined variables, and must return an image in a format that Cloudinary supports for transformations.
  3. Any other Cloudinary transformations given are applied to the result of the preprocessing function: the custom_pre_function parameter is applied first, no matter its location in the transformation chain.

For example, to deliver the 'sample' image after preprocessing the remote function located at 'https://my.preprocess.custom/function':

Ruby:
cl_image_tag("sample.jpg", 
  :sign_url => true,
  :custom_pre_function => {
    :function_type => "remote", 
    :source => "https://my.preprocess.custom/function"})
PHP:
cl_image_tag("sample.jpg", array(
  "sign_url" => true,
  "custom_pre_function" => array(
    "function_type" => "remote", 
    "source" => "https://my.preprocess.custom/function")))
Python:
CloudinaryImage("sample.jpg").image(
  sign_url = True,
  custom_pre_function = {
    "function_type" = "remote", 
    "source" = "https://my.preprocess.custom/function"})
Node.js:
cloudinary.image("sample.jpg", {
  sign_url: true,
  custom_pre_function:{
    function_type: "remote", 
    source: "https://my.preprocess.custom/function"}})
Java:
cloudinary.url().transformation(new Transformation()
    .signed(true)
    .customPreFunction(remote("https://my.preprocess.custom/function")))
  .imageTag("sample.jpg");
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
    .Signed(true)
    .customPreFunction(CustomFunction.Remote("https://my.preprocess.custom/function")))
  .BuildImageTag("sample.jpg")

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/s--994c2b72--/fn_pre:remote:aHR0cHM6Ly9teS5wcmVwcm9jZXNzLmN1c3RvbS9mdW5jdGlvbg==/sample.jpg