Inject Your Own Image Processing by Using a Custom Function

Cloudinary offers a wide array of image transformations and effects to apply to images as part of our image-processing pipeline, helping to ensure that your images fit the graphic design of your website or mobile application. Cloudinary is an open platform, and you can use our APIs, Widgets and UI to build the media management flow that matches your needs.

But despite the fact that our extensive and versatile range of transformations is continuously growing with new transformations added on a regular basis, the list is not complete by any means. There are just too many ways that an image could potentially be transformed to be able to offer every possible transformation.

Cloudinary strives to provide a solution that is as complete as possible, catering to the needs of thousands of customers. That said, when catering for so many customers, having a plethora of needs, there can be instances where a customer needs a capability that is not yet supported. At Cloudinary we try to fill our customers needs, but may not always be able to implement new functionality in a timely manner, or there just may be no other demand for a customer’s highly specialized requirements requiring some very specific code to implement.

To this end, Cloudinary now supports injecting custom functions into the image transformation pipeline, allowing our customers to implement their own transformations on their images. This new functionality takes our image processing capabilities to the next level, to be fully extensible and open – allowing you to add your own processing capabilities.

For example, if you wanted to limit a delivered image to the 3 most predominant colors in the image, you could add a custom function to make this happen.

There are 3 ways to inject a function with the custom_function parameter (fn in URLs):

• Use a WebAssembly function uploaded as a compiled .wasm file to your account.
• Supply an external URL as the source of a remote function/lambda.
• Inject a remote preprocessing function before Cloudinary does any processing on the file.

A WebAssembly function is a self-contained compiled WASM file that will run on Cloudinary’s servers as part of the image transformation pipeline. The function is provided with a pointer to an RGBA interleaved pixel buffer together with a pointer to any metadata (context, tags and variables), and must return a pointer to an output buffer containing the image pixels (optionally including metadata as well).

The function cannot reference any outside URLs, so this case is most useful for adding new functionality (a new transformation) that needs to run quickly on our servers. This is the best way to use a special effect or add a new filter.

The WebAssembly function must be uploaded to your account as a raw authenticated asset, and then referenced in a custom function: use the custom_function parameter with the function_type set to “wasm” (fn_wasm in URLs), and the source parameter set to the public_id of your compiled WASM file.

For example, to deliver the ‘rainbow’ image after running the WebAssembly function located in a compiled WASM file uploaded to your account with the public_id of ‘blur.wasm’:

Loading code examples

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/fn_wasm:blur.wasm/rainbow.jpg
Code language: JavaScript (javascript)

The following interactive example uses a custom function to add a ‘quantize’ effect to an image (just like the example in the introduction above), where the image is limited to a specific number of colors that is supplied as an additional variable parameter. This could be important for displaying images on devices that support a limited number of colors and for efficiently compressing certain kinds of images.

1  Select the number of colors:    

2  Pick a target photo, or upload your own:    Upload an image 3  Press the magic button!

A remote function can pretty much do anything with the PNG image it is sent for processing, only limited by the programming language used and your imagination. Your remote function could access external databases for extracting information, add new layers and effects, or even incorporate remote calls to 3rd party APIs. This is a great way to incorporate any external functionality into the pipeline, in effect adding your own “add-on” to your toolbox of image transformations.

Remote functions are especially useful for new Cloudinary customers that may already be using 3rd party tools to accomplish some of their image transformations, whether it’s something Cloudinary doesn’t currently support, or just something that may give slightly different results with a Cloudinary transformation, for example, a 3rd party image enhancement service. You can host your own remote functions or just use a serverless computing vendor such as AWS Lambda or Google Cloud Functions to manage your custom function.

A remote custom function receives an input image in PNG format along with metadata (context, tags and variables), and must return an image in a format that Cloudinary supports (optionally including metadata as well). Use the custom_function parameter with the function_type set to “remote” (fn_remote in URLs), and the source parameter set to the URL of the custom function. As a security precaution, and to prevent unmoderated use of your remote function, the delivery URL also needs to be signed, which simply means adding the sign_url parameter set to “true” to the SDK method.

For example, to deliver the ‘rainbow’ image after running the remote function located at ‘https://my.example.custom/function’:

Loading code examples

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/s--21ba7cb4--/fn_remote:aHR0cHM6Ly9teS5leGFtcGxlLmN1c3RvbS9mdW5jdGlvbg==//rainbow.jpg
Code language: JavaScript (javascript)

The custom functions discussed so far are given an image for processing, either a pointer to an image bitmap for WASM functions, or a PNG image for remote functions. However, Cloudinary also allows you to insert a remote custom function earlier in the transformation chain, before any processing is done on the image at all. Whereas the remote function option described above is sent an image in PNG format, a preprocessing remote function is sent the original image file, as uploaded to Cloudinary. This may be necessary in two scenarios:

1. The original image file is in a format that Cloudinary only supports for upload, but not for transformations without first converting to another format, for example the SVG format.
2. Cloudinary only offers basic transformations for the given file format, for example the PDF format. If you want to extract or replace text within the file, rearrange or delete specific layers, or any other advanced editing option, you can integrate your preprocessing remote function with a 3rd party tool for this very purpose.

Use the custom_pre_function parameter (fn_pre in URLs) to call the custom function to preprocess, with the function_type set to “remote” (fn_pre:remote in URLs), and the source parameter set to the URL of the custom function.

For example, to deliver the ‘logo’ image after running the remote function located at ‘https://my.preprocess.custom/function’:

Loading code examples

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/s--511a3ca4--/fn_pre:remote:aHR0cHM6Ly9teS5leGFtcGxlLmN1c3RvbS9mdW5jdGlvbg==//logo.jpg
Code language: JavaScript (javascript)

The following example uses variables together with a preprocessing remote function in order to render a text string within an SVG file. The variables set the colors for the text (fillc) and the font stroke (strokec), as well as the text itself (msgstr). Note that the uploaded SVG file has been prepared to include variables, each of them specified with triple hash marks before and after the variable name (e.g., ###fillc###). The custom preprocessing remote function searches for these variables and replaces them with the given values.

Exhibit 1 – The SVG file uploaded to Cloudinary including the variable definitions:

<svg viewBox="0 0 1000 300"
  xmlns="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink">
  <defs>
    <path id="MyPath"
      d="M 100 200
      C 200 100 300   0 400 100
      C 500 200 600 300 700 200
      C 800 100 900 100 900 100" />
  </defs>
  <text x="10" y="100" font-size="40" fill="###fillc###" stroke="###strokec###">
    <textPath xlink:href="#MyPath">
       ###msgstr###
    </textPath>
  </text>
</svg>
Code language: HTML, XML (xml)

Exhibit 2 – The SDK code for generating the image tag:

Loading code examples

Exhibit 3 – The code generates a URL similar to:

http://res.cloudinary.com/demo/image/upload/s--kkSWOzr4--/$fillc_!yellow!,$strokec_!black!,$msgstr_!The quick brown fox jumps over the lazy dog!,fn_pre:remote:aHR0cHM6Ly91cm1haTB1cDFqLmV4ZWN1dGUtYXBpLnVzLWVhc3QtMS5hbWF6b25hd3MuY29tL3Rlc3Q=/msg_tmp.svg
Code language: PHP (php)

Exhibit 4 – The resulting image:

<img src=“https://res.cloudinary.com/demo/image/upload/msg_tmp_comp.jpg”

Cloudinary strives to enhance our core features as best we can, prioritizing features that will benefit many customers, and to be as responsive as possible to customer requests. As extensive as our open platform functionality may be, we may not currently support a specific functionality, and so custom functions give our customers the flexibility to incorporate their own functionality in the image-processing pipeline. It also makes it relatively simple to incorporate other 3rd party functionality. Make sure to read up on all the details about custom functions in the documentation, and note that custom functions are available with no extra charge for all Cloudinary’s plans, including the free plan.