Custom functions

Cloudinary supports injecting a custom function into the image transformation pipeline. You can either use a remote function/lambda as your source, or run WebAssembly functions from a compiled wasm file uploaded to your account.

To specify a custom function to call, use the custom_function parameter (fn in URLs). The parameter accepts an object detailing the function to inject as follows:

Parameter Description
function_type The type of function to run, either 'remote' or 'wasm'
source The source of the custom function, either the public_id of the wasm file or the URL of the remote function

Tip
Once requested, derived images from custom functions are cached like any other derived images generated by Cloudinary: modifying your custom function will not generate a new image for the same dynamic URL as the URL itself has not changed. If you need to bypass this issue when you change a custom function, we recommend adding a version component to the Cloudinary URL.

WebAssembly functions

Compiled wasm files may be uploaded as raw authenticated resources to your Cloudinary account and then referenced in a custom function. Use the custom_function parameter with the function_type set to "wasm" (fn_wasm in URLs), and the source parameter set to the public_id of your compiled wasm file. If the public ID includes a folder component, replace the slash with a colon (e.g., custom/example.wasm becomes custom:example.wasm).

For example, to deliver the 'sample' image after running the WebAssembly functions located in a compiled wasm file:

  1. Upload your compiled wasm file to your account as an authenticated raw file:

    Ruby:
    Copy to clipboard
    Cloudinary::Uploader.upload("my_example.wasm", 
      :use_filename => true,
      :type => :authenticated,
      :resource_type => :raw)
    PHP:
    Copy to clipboard
    \Cloudinary\Uploader::upload("my_example.wasm", 
      array(
        "use_filename" => TRUE,
        "type" => "authenticated",
        "resource_type" => "raw"));
    Python:
    Copy to clipboard
    cloudinary.uploader.upload("my_example.wasm", 
      use_filename = True,
      type = "authenticated",
      resource_type = "raw")
    Node.js:
    Copy to clipboard
    cloudinary.v2.uploader.upload("my_example.wasm", 
      { use_filename: true,
        type: "authenticated",
        resource_type: "raw" }, 
      function(error, result) {console.log(result, error); });
    Java:
    Copy to clipboard
    cloudinary.uploader().upload("my_example.wasm", 
      ObjectUtils.asMap(
        "use_filename", "true",
        "type", "authenticated",
        "resource_type", "raw"));
    .Net:
    Copy to clipboard
    var uploadParams = new RawUploadParams(){  // by default, ResourceType is already set to "raw"
      UseFilename = true,
      Type = "authenticated",
      File = new FileDescription(@"my_example.wasm")};
    var uploadResult = cloudinary.Upload(uploadParams);
    CLI:
    Copy to clipboard
    cld uploader upload my_example.wasm use_filename=true type=authenticated resource_type=raw
  2. Deliver the sample image after running the WebAssembly functions located in your now uploaded my_example.wasm file:

    Ruby:
    Copy to clipboard
    cl_image_tag("sample.jpg",
      :custom_function => {
        :function_type => "wasm", 
        :source => "my_example.wasm"})
    PHP:
    Copy to clipboard
    cl_image_tag("sample.jpg", array(
      "custom_function" => array(
        "function_type" => "wasm", 
        "source" => "my_example.wasm")))
    Python:
    Copy to clipboard
    CloudinaryImage("sample.jpg").image(
      custom_function = {
        "function_type" = "wasm", 
        "source" = "my_example.wasm"})
    Node.js:
    Copy to clipboard
    cloudinary.image("sample.jpg", {
      custom_function:{
        function_type: "wasm", 
        source: "my_example.wasm"}})
    Java:
    Copy to clipboard
    cloudinary.url().transformation(new Transformation()
      .customFunction(wasm("my_example.wasm")))
      .imageTag("sample.jpg");
    JS:
    Copy to clipboard
    cloudinary.imageTag('sample.jpg', {
      customFunction: new cloudinary.CustomFunction()
        .functionType("wasm")
        .source("my_example.wasm")}).toHtml();
    jQuery:
    Copy to clipboard
    $.cloudinary.image('sample.jpg', {
      customFunction: new cloudinary.CustomFunction()
        .functionType("wasm")
        .source("my_example.wasm")})
    React:
    Copy to clipboard
    <Image publicId="sample.jpg">
      <Transformation customFunction={{
        functionType: "wasm",
        source: "my_example.wasm"}} />
    </Image>
    Angular:
    Copy to clipboard
    <cl-image public-id="sample.jpg">
      <cl-transformation customFunction={
       "function_type": "wasm",
        "source": "my_example.wasm"}>
      </cl-transformation>
    </cl-image>
    .Net:
    Copy to clipboard
    cloudinary.Api.UrlImgUp.Transform(new Transformation()
      .CustomFunction(CustomFunction.Wasm("my_example.wasm")))
      .BuildImageTag("sample.jpg")
    Android:
    Copy to clipboard
    MediaManager.get().url().transformation(new Transformation()
      .customFunction(new CustomFunction()
        .functionType("wasm")
        .source("my_example.wasm"))
      .generate("sample.jpg");
    iOS:
    Copy to clipboard
    imageView.cldSetImage(cloudinary.createUrl()
      .setTransformation(CLDTransformation()
        .setCustomFunction(CLDCustomFunction()
          .setFunctionType("wasm")
          .setSource("my_example.wasm")))
      .generate("sample.jpg")!, cloudinary: cloudinary)

    The code generates a URL similar to:

    Copy to clipboard
    https://res.cloudinary.com/demo/image/upload/fn_wasm:my_example.wasm/sample.jpg

WebAssembly contract

Note
The samples below are given in the Rust programming language which is then compiled to a wasm target - but this can be achieved with any language that compiles to .wasm.

The WebAssembly functions in your compiled wasm file need to comply with a specific interface - you must provide the following 3 public functions:

1. alloc

Your alloc method needs to allocate memory according to the size given and then return a pointer to the allocated memory.

Copy to clipboard
    alloc(size: usize) -> *mut u8

2. dealloc

Your dealloc method should deallocate memory at the given pointer, according to the size given.

Copy to clipboard
    dealloc(ptr: *mut u8, cap: usize)

3. transform

Your transform method should include the code you want to perform the actual transformation of the image. The method receives the image width, image height, a pointer to the pixel buffer (of size = width x height x 4) where the pixel scheme is guaranteed to be RGBA interleaved, a pointer to the metadata, and the metadata size.

Metadata is given as a JSON structure which contains:

Copy to clipboard
{
  context: [Map of string keys to string values],
  tags: [Array of strings],
  variables: [Map of string keys to string values]
}
  • context - Context metadata defined for the asset.
  • tags - Tags applied to the asset.
  • variables - Any user-defined variables specified as part of the transformation.

You can pass files to WebAssembly functions as base64 encoded strings using file reference variables, as shown in this example.

The transform function must return a pointer to the output buffer that contains the following information: width as a 32 bit BigEndian, followed by the height as a 32 bit BigEndian, followed by the image pixel buffer (RGBA scheme again).

Copy to clipboard
    transform(width: u32, height: u32, image_ptr: *mut u8, meta_ptr: *mut u8, meta_size: usize) -> u32

WebAssembly example

The following example applies a blur effect to an image:

Copy to clipboard
#![feature(exact_chunks)]

#[macro_use]
extern crate serde_derive;
extern crate image;
extern crate serde;
extern crate serde_json;
extern crate byteorder;

use std::mem;
use byteorder::{ WriteBytesExt, BigEndian};
use image::{RgbaImage, imageops};

#[derive(Deserialize, Debug)]
struct Metadata {
    context: Option<std::collections::HashMap<String, String>>,
    tags: Option<Vec<String>>,
    variables: Option<std::collections::HashMap<String, i32>>,
}

#[no_mangle]
pub extern "C" fn alloc(size: usize) -> *mut u8 {
    let mut buf = Vec::<u8>::with_capacity(size);
    let ptr = buf.as_mut_ptr();
    mem::forget(buf);
    return ptr;
}

#[no_mangle]
pub extern "C" fn dealloc(ptr: *mut u8, cap: usize) {
    unsafe  {
      let _buf = Vec::from_raw_parts(ptr, 0, cap);
    }
}

#[no_mangle]
pub extern "C" fn transform(width: u32, height: u32, image_ptr: *mut u8, meta_ptr: *mut u8, meta_size: usize) -> u32 {
  let size = (width * height * 4) as usize;
  let bytes = unsafe {Vec::from_raw_parts(image_ptr, size, size)};
  let meta_bytes = unsafe {Vec::from_raw_parts(meta_ptr, meta_size, meta_size)};
  let metadata: Metadata = serde_json::from_slice(&meta_bytes).expect("Failed to deserialize metadata json");
  host_trace(format!("{:?}", metadata));
  let (out_w, out_h, mut out_buffer) = blur(width, height, bytes, metadata);
  let mut dims = vec![];
  let _ = dims.write_u32::<BigEndian>(out_w);
  let _ = dims.write_u32::<BigEndian>(out_h);
  dims.append(&mut out_buffer);
  let out_buffer = dims;
  let out_ptr = out_buffer.as_ptr() as u32;
  mem::forget(out_buffer);
  out_ptr  
}
fn host_trace(x: String) {
  let buf = x.into_bytes();
  let length = buf.len();
  let ptr = buf.as_ptr();
  unsafe { trace(ptr as u32, length as u32) }
}
extern "C" {
  pub fn trace(x: u32, length: u32);
}
fn blur(width: u32, height: u32, bytes: Vec<u8>, _metadata: Metadata) -> (u32, u32, Vec<u8>) {
  let ref img = RgbaImage::from_raw(width, height, bytes).unwrap();
  let subimg = imageops::blur(img, 2.5);
  let out_w = subimg.width();
  let out_h = subimg.height();
  let out_buffer = subimg.into_raw();
  (out_w, out_h, out_buffer)
}

Remote functions

You can call a remote function/lambda as part of the transformation chain. The remote function receives an input image file (PNG) along with metadata, and must return an image file (optionally along with metadata also). Use the custom_function parameter with the function_type set to "remote" (fn_remote in URLs), and the source parameter set to the URL of the custom function. The delivery URL also needs to be signed, which means also adding the sign_url parameter set to "true" to the SDK method.

Note
The image is sent in the PNG format to the remote function, even if the uploaded image is in a different format.

For example, to deliver the 'sample' image after running the remote function located at 'https://my.example.custom/function':

Ruby:
Copy to clipboard
cl_image_tag("sample.jpg", 
  :sign_url => true,
  :custom_function => {
    :function_type => "remote", 
    :source => "https://my.example.custom/function"})
PHP:
Copy to clipboard
cl_image_tag("sample.jpg", array(
  "sign_url" => true,
  "custom_function" => array(
    "function_type" => "remote", 
    "source" => "https://my.example.custom/function")))
Python:
Copy to clipboard
CloudinaryImage("sample.jpg").image(
  sign_url = True,
  custom_function = {
    "function_type" = "remote", 
    "source" = "https://my.example.custom/function"})
Node.js:
Copy to clipboard
cloudinary.image("sample.jpg", {
  sign_url: true,
  custom_function:{
    function_type: "remote", 
    source: "https://my.example.custom/function"}})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation()
    .signed(true)
    .customFunction(remote("https://my.example.custom/function")))
  .imageTag("sample.jpg");
.Net:
Copy to clipboard
cloudinary.Api.UrlImgUp.Transform(new Transformation()
    .Signed(true)
    .CustomFunction(CustomFunction.Remote("https://my.example.custom/function")))
  .BuildImageTag("sample.jpg")

The code generates a URL similar to:

Copy to clipboard
https://res.cloudinary.com/demo/image/upload/s--89bc7b34--/fn_remote:aHR0cHM6Ly9teS5leGFtcGxlLmN1c3RvbS9mdW5jdGlvbg==/sample.jpg

Note
The Cloudinary SDKs automatically generate a base 64 encoded URL from the source parameter: you will need to supply the source URL in base 64 with padding if you generate the delivery URL in your own code.

Request structure

The URL of your remote function receives the following information in the HTTP POST request from Cloudinary:

Request headers:

  • X-Cld-Timestamp - an integer value representing the time the request was sent in Unix time.
  • X-Cld-Signature - a signature string for verification: the string is an SHA-1 Hex-Digest of the request body + timestamp + your API secret.

Request body:

  • file - the (binary) image file.
  • metadata - a JSON structure that includes the current_page, tags, coordinates (each key represents a coordinate source) and variables (key-value pairs representing variable names and values). For example:

    Copy to clipboard
    {
      "current_page": 1,
      "tags": [],
      "coordinates": {
        "eyedea": {
          "coords": [],
          "kind": "eyedea",
          "on_original": true,
          "failed": false
        }
      },
      "variables": {
        "z": 5,
        "foo": 10
      }
    }

Response structure

Your response should include the image Content-Type in the header and the image data in the body. For example, if using API Gateway to host your remote function then the response would be similar to:

Copy to clipboard
{
  "statusCode": 200,
  "headers": {
    "Content-Type": "image/jpeg",
    "Content-Length": "result.length"
  },
  "isBase64Encoded": true,             // base64 is restriction imposed by API Gateway
  "body": "result.toString('base64')"   // image data encoded as base64 with padding
}

The body of your response can contain just the image data or you can include new metadata as follows:

  1. Start the response body with the string 'CLDB'.
  2. Add 4 bytes (BigEndian) describing the byte length of the result image buffer.
  3. Add the image buffer.
  4. Add 4 bytes (BigEndian) describing the byte length of the result metadata JSON buffer.
  5. Add the metadata buffer.
Copy to clipboard
CLDB + length_of_image_buffer + image_buffer + length_of_metadata buffer + metadata_buffer

Sample remote function

The following code is an example of a remote function that is hosted on AWS lambda behind an API gateway and is written in JavaScript. The function resizes an image to a width of 314px and adds a text overlay of the current date (subsequent requests will retrieve this image from the CDN cache and the date will not change). It also returns new metadata:

Copy to clipboard
const im = require('imagemagick');
const multipart = require('aws-lambda-multipart-parser');
const fs = require('fs');
const fail = (message) => {
    console.log(message);
    throw new Error(message);
};
const perform = (operation, args) => new Promise((resolve, reject) => im[operation](args, (err, res) => {
    if (err) {
        console.log(`${operation} operation failed:`, err);
        reject(err);
    } else {
        console.log(`${operation} completed successfully`);
        resolve(res);
    }
}));
const postProcessResource = (resource, fn) => {
    let ret = null;
    if (resource) {
        if (fn) {
            ret = fn(resource);
        }
        try {
            fs.unlinkSync(resource);
        } catch (err) {
            // Ignore
        }
    }
    return ret;
};
const transform = async (file) => {
    // current time as string
    const date = (new Date()).toDateString();
    // transformation in imagemagick: resize to 314px, overlay text at x=5px, y=20px.
    const customArgs = ['-resize', '314x', '-fill', 'blue', '-draw', `text 5,20 '${date}'`];
    // prepare input and output files
    let inputFile = null;
    let outputFile = null;
    inputFile = `/tmp/inputFile`;
    fs.writeFileSync(inputFile, file);
    customArgs.unshift(inputFile);
    outputFile = `/tmp/outputFile.jpg`;
    customArgs.push(outputFile);
    // actual conversion
    const output = await perform('convert', customArgs);
    postProcessResource(inputFile);
    if (outputFile) {
        return postProcessResource(outputFile, (file) => new Buffer(fs.readFileSync(file)));
    }
    // Return the command line output as a debugging aid
    return output;
};
exports.handler = async (event, context, callback) => {
    const parsedRequest = multipart.parse(event, false);
    const file = parsedRequest.file.content;
    return transform(file).then((result) => {
      // return the image and new metadata.   
      if (event.queryStringParameters && event.queryStringParameters.cldb) {
          const bodyLengthBuf = new Buffer(4);
          const bodyLength = result.length;
          bodyLengthBuf.writeUInt32BE(bodyLength);
          const metadata = Buffer.from(JSON.stringify({"coordinates": {"custom": [[45,57,100,120]]}}));
          const metadataLengthBuf = new Buffer(4);
          const metadataLength = metadata.length;
          metadataLengthBuf.writeUInt32BE(metadataLength);
          result = Buffer.concat([Buffer.from('CLDB'), bodyLengthBuf, result, metadataLengthBuf, metadata], 3*4 + metadataLength + bodyLength);
      }
      callback(null, {
        statusCode: 200,
        headers: { 'Content-Type': 'image/jpeg', 'Content-Length': result.length},
        isBase64Encoded: true,
        body: result.toString('base64')
      });
    }).catch((error) => {
      callback(null, {
        statusCode: 502,
        headers:    { 'Content-Type': 'application/json' },
        body:    `{"error": "Error manipulating image ${error}"}`
      });
    });
};

Preprocessing custom functions

You can insert your custom function earlier in the transformation chain, before Cloudinary does any processing whatsoever on the image. Whereas the remote function option described above is sent an image in PNG format, a preprocessing remote function is sent the original image file, as uploaded to Cloudinary. For example, you can upload images in a format Cloudinary does not support for transformations and use a custom function to return an image format that Cloudinary does support.

Only Remote Functions are supported for preprocessing as described above, except for the following differences:

  1. Use the custom_pre_function parameter (fn_pre in URLs) to call the custom function to preprocess. The parameter accepts the same type of object as the custom_function parameter, detailing the function_type ('remote') and source.
  2. The preprocessing function is sent the original unaltered image, plus any defined variables, and must return an image in a format that Cloudinary supports for transformations.
  3. Any other Cloudinary transformations given are applied to the result of the preprocessing function: the custom_pre_function parameter is applied first, no matter its location in the transformation chain.

For example, to deliver the 'sample' image after preprocessing the remote function located at 'https://my.preprocess.custom/function':

Ruby:
Copy to clipboard
cl_image_tag("sample.jpg", 
  :sign_url => true,
  :custom_pre_function => {
    :function_type => "remote", 
    :source => "https://my.preprocess.custom/function"})
PHP:
Copy to clipboard
cl_image_tag("sample.jpg", array(
  "sign_url" => true,
  "custom_pre_function" => array(
    "function_type" => "remote", 
    "source" => "https://my.preprocess.custom/function")))
Python:
Copy to clipboard
CloudinaryImage("sample.jpg").image(
  sign_url = True,
  custom_pre_function = {
    "function_type" = "remote", 
    "source" = "https://my.preprocess.custom/function"})
Node.js:
Copy to clipboard
cloudinary.image("sample.jpg", {
  sign_url: true,
  custom_pre_function:{
    function_type: "remote", 
    source: "https://my.preprocess.custom/function"}})
Java:
Copy to clipboard
cloudinary.url().transformation(new Transformation()
    .signed(true)
    .customPreFunction(remote("https://my.preprocess.custom/function")))
  .imageTag("sample.jpg");
.Net:
Copy to clipboard
cloudinary.Api.UrlImgUp.Transform(new Transformation()
    .Signed(true)
    .customPreFunction(CustomFunction.Remote("https://my.preprocess.custom/function")))
  .BuildImageTag("sample.jpg")

The code generates a URL similar to:

Copy to clipboard
https://res.cloudinary.com/demo/image/upload/s--994c2b72--/fn_pre:remote:aHR0cHM6Ly9teS5wcmVwcm9jZXNzLmN1c3RvbS9mdW5jdGlvbg==/sample.jpg

✔️ Feedback sent!

Rate this page: