Cloudinary Blog

Inject Your Own Image Processing by Using a Custom Function

Use a custom function in the image delivery pipeline

Cloudinary offers a wide array of image manipulations and effects to apply to images as part of our image-processing pipeline, helping to ensure that your images fit the graphic design of your website or mobile application. Cloudinary is an open platform, and you can use our APIs, Widgets and UI to build the media management flow that matches your needs.

But despite the fact that our extensive and versatile range of transformations is continuously growing with new transformations added on a regular basis, the list is not complete by any means. There are just too many ways that an image could potentially be manipulated to be able to offer every possible transformation.

Cloudinary strives to provide a solution that is as complete as possible, catering to the needs of thousands of customers. That said, when catering for so many customers, having a plethora of needs, there can be instances where a customer needs a capability that is not yet supported. At Cloudinary we try to fill our customers needs, but may not always be able to implement new functionality in a timely manner, or there just may be no other demand for a customer's highly specialized requirements requiring some very specific code to implement.

To this end, Cloudinary now supports injecting custom functions into the image manipulation pipeline, allowing our customers to implement their own manipulations on their images. This new functionality takes our image processing capabilities to the next level, to be fully extensible and open - allowing you to add your own processing capabilities.

For example, if you wanted to limit a delivered image to the 3 most predominant colors in the image, you could add a custom function to make this happen.

quantized example

There are 3 ways to inject a function with the custom_function parameter (fn in URLs):

Use a WebAssembly Function

A WebAssembly function is a self-contained compiled WASM file that will run on Cloudinary's servers as part of the image transformation pipeline. The function is provided with a pointer to an RGBA interleaved pixel buffer together with a pointer to any metadata (context, tags and variables), and must return a pointer to an output buffer containing the image pixels (optionally including metadata as well).

The function cannot reference any outside URLs, so this case is most useful for adding new functionality (a new transformation) that needs to run quickly on our servers. This is the best way to use a special effect or add a new filter.

The WebAssembly function must be uploaded to your account as a raw authenticated asset, and then referenced in a custom function: use the custom_function parameter with the function_type set to "wasm" (fn_wasm in URLs), and the source parameter set to the public_id of your compiled WASM file.

For example, to deliver the 'rainbow' image after running the WebAssembly function located in a compiled WASM file uploaded to your account with the public_id of 'blur.wasm':

Ruby:
cl_image_tag("rainbow.jpg",
      :custom_function => {
        :function_type => "wasm", 
        :source => "blur.wasm"})
PHP:
cl_image_tag("rainbow.jpg", array(
      "custom_function" => array(
        "function_type" => "wasm", 
        "source" => "blur.wasm")))
Python:
CloudinaryImage("rainbow.jpg").image(
      custom_function = {
        "function_type" = "wasm", 
        "source" = "blur.wasm"})
Node.js:
cloudinary.image("rainbow.jpg", {
      custom_function:{
        function_type: "wasm", 
        source: "blur.wasm"}})
Java:
cloudinary.url().transformation(new Transformation()
      .customFunction(wasm("blur.wasm")))
      .imageTag("rainbow.jpg");
JS:
cloudinary.imageTag('rainbow.jpg', {
      customFunction: new cloudinary.CustomFunction()
        .functionType("wasm")
        .source("blur.wasm")}).toHtml();
jQuery:
$.cloudinary.image('rainbow.jpg', {
      customFunction: new cloudinary.CustomFunction()
        .functionType("wasm")
        .source("blur.wasm")})
React:
<Image publicId="rainbow.jpg">
      <Transformation customFunction={{
        functionType: "wasm",
        source: "blur.wasm"}} />
    </Image>
Angular:
<cl-image public-id="rainbow.jpg">
      <cl-transformation customFunction={
       functionType: "wasm",
        source: "blur.wasm"}>
      </cl-transformation>
    </cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
      .CustomFunction(CustomFunction.Wasm("blur.wasm")))
      .BuildImageTag("rainbow.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
      .customFunction(new CustomFunction()
        .functionType("wasm")
        .source("blur.wasm"))
      .generate("rainbow.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl()
      .setTransformation(CLDTransformation()
        .setCustomFunction(CLDCustomFunction()
          .setFunctionType("wasm")
          .setSource("blur.wasm")))
      .generate("rainbow.jpg")!, cloudinary: cloudinary)

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/fn_wasm:blur.wasm/rainbow.jpg

Interactive Example

The following interactive example uses a custom function to add a 'quantize' effect to an image (just like the example in the introduction above), where the image is limited to a specific number of colors that is supplied as an additional variable parameter. This could be important for displaying images on devices that support a limited number of colors and for efficiently compressing certain kinds of images.

1  Select the number of colors:    

2  Pick a target photo, or upload your own:   
golf course target photo coffee target photo old man target photo custom target photo
3  Press the magic button!

Call a Remote Function

A remote function can pretty much do anything with the PNG image it is sent for processing, only limited by the programming language used and your imagination. Your remote function could access external databases for extracting information, add new layers and effects, or even incorporate remote calls to 3rd party APIs. This is a great way to incorporate any external functionality into the pipeline, in effect adding your own "add-on" to your toolbox of image transformations.

Remote functions are especially useful for new Cloudinary customers that may already be using 3rd party tools to accomplish some of their image transformations, whether it's something Cloudinary doesn't currently support, or just something that may give slightly different results with a Cloudinary transformation, for example, a 3rd party image enhancement service. You can host your own remote functions or just use a serverless computing vendor such as AWS Lambda or Google Cloud Functions to manage your custom function.

A remote custom function receives an input image in PNG format along with metadata (context, tags and variables), and must return an image in a format that Cloudinary supports (optionally including metadata as well). Use the custom_function parameter with the function_type set to "remote" (fn_remote in URLs), and the source parameter set to the URL of the custom function. As a security precaution, and to prevent unmoderated use of your remote function, the delivery URL also needs to be signed, which simply means adding the sign_url parameter set to "true" to the SDK method.

For example, to deliver the 'rainbow' image after running the remote function located at 'https://my.example.custom/function':

Ruby:
cl_image_tag("rainbow.jpg",
      :sign_url => true,
      :custom_function => {
        :function_type => "remote", 
        :source => "https://my.example.custom/function"})
PHP:
cl_image_tag("rainbow.jpg", array(
      "sign_url" => true,
      "custom_function" => array(
        "function_type" => "remote", 
        "source" => "https://my.example.custom/function")))
Python:
CloudinaryImage("rainbow.jpg").image(
      sign_url = True,
      custom_function = {
        "function_type" = "remote", 
        "source" = "https://my.example.custom/function"})
Node.js:
cloudinary.image("rainbow.jpg", {
      sign_url: true,
      custom_function:{
        function_type: "remote", 
        source: "https://my.example.custom/function"}})
Java:
cloudinary.url().transformation(new Transformation()
      .signed(true)
      .customFunction(remote("https://my.example.custom/function")))
      .imageTag("rainbow.jpg");
JS:
cloudinary.imageTag('rainbow.jpg', {
      signUrl: true,
      customFunction: new cloudinary.CustomFunction()
        .functionType("remote")
        .source("https://my.example.custom/function")}).toHtml();
jQuery:
$.cloudinary.image('rainbow.jpg', {
      signUrl: true,
      customFunction: new cloudinary.CustomFunction()
        .functionType("remote")
        .source("https://my.example.custom/function")})
React:
<Image publicId="rainbow.jpg" signUrl="true">
      <Transformation customFunction={{
        functionType: "remote",
        source: "https://my.example.custom/function"}} />
    </Image>
Angular:
<cl-image public-id="rainbow.jpg" sign-url="true">
      <cl-transformation customFunction={
       functionType: "remote",
       source: "https://my.example.custom/function"}>
      </cl-transformation>
    </cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
      .Signed(true)
      .CustomFunction(CustomFunction.Remote("https://my.example.custom/function")))
      .BuildImageTag("rainbow.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
      .signed(true)
      .customFunction(new CustomFunction()
        .functionType("remote")
        .source("https://my.example.custom/function"))
      .generate("rainbow.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl()
      .setTransformation(CLDTransformation()
        .setCustomFunction(CLDCustomFunction()
          .setFunctionType("remote")
          .setSource("https://my.example.custom/function")))
      .generate("rainbow.jpg", signUrl: true)!, cloudinary: cloudinary)

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/s--21ba7cb4--/fn_remote:aHR0cHM6Ly9teS5leGFtcGxlLmN1c3RvbS9mdW5jdGlvbg==//rainbow.jpg

Preprocess an Image

The custom functions discussed so far are given an image for processing, either a pointer to an image bitmap for WASM functions, or a PNG image for remote functions. However, Cloudinary also allows you to insert a remote custom function earlier in the transformation chain, before any processing is done on the image at all. Whereas the remote function option described above is sent an image in PNG format, a preprocessing remote function is sent the original image file, as uploaded to Cloudinary. This may be necessary in two scenarios:

  1. The original image file is in a format that Cloudinary only supports for upload, but not for transformations without first converting to another format, for example the SVG format.
  2. Cloudinary only offers basic transformations for the given file format, for example the PDF format. If you want to extract or replace text within the file, rearrange or delete specific layers, or any other advanced editing option, you can integrate your preprocessing remote function with a 3rd party tool for this very purpose.

Use the custom_pre_function parameter (fn_pre in URLs) to call the custom function to preprocess, with the function_type set to "remote" (fn_pre:remote in URLs), and the source parameter set to the URL of the custom function.

For example, to deliver the 'logo' image after running the remote function located at 'https://my.preprocess.custom/function':

Ruby:
cl_image_tag("logo.jpg",
      :sign_url => true,
      :custom_pre_function => {
        :function_type => "remote", 
        :source => "https://my.example.custom/function"})
PHP:
cl_image_tag("logo.jpg", array(
      "sign_url" => true,
      "custom_pre_function" => array(
        "function_type" => "remote", 
        "source" => "https://my.example.custom/function")))
Python:
CloudinaryImage("logo.jpg").image(
      sign_url = True,
      custom_pre_function = {
        "function_type" = "remote", 
        "source" = "https://my.example.custom/function"})
Node.js:
cloudinary.image("logo.jpg", {
      sign_url: true,
      custom_pre_function:{
        function_type: "remote", 
        source: "https://my.example.custom/function"}})
Java:
cloudinary.url().transformation(new Transformation()
      .signed(true)
      .customPreFunction(remote("https://my.example.custom/function")))
      .imageTag("logo.jpg");
JS:
cloudinary.imageTag('logo.jpg', {
      signUrl: true,
      customPreFunction: new cloudinary.CustomFunction()
        .functionType("remote")
        .source("https://my.example.custom/function")}).toHtml();
jQuery:
$.cloudinary.image('logo.jpg', {
      signUrl: true,
      customPreFunction: new cloudinary.CustomFunction()
        .functionType("remote")
        .source("https://my.example.custom/function")})
React:
<Image publicId="logo.jpg" signUrl="true">
      <Transformation customPreFunction={{
        functionType: "remote",
        source: "https://my.example.custom/function"}} />
    </Image>
Angular:
<cl-image public-id="logo.jpg" sign-url="true">
      <cl-transformation customPreFunction={
       functionType: "remote",
       source: "https://my.example.custom/function"}>
      </cl-transformation>
    </cl-image>
.Net:
cloudinary.Api.UrlImgUp.Transform(new Transformation()
      .Signed(true)
      .CustomFunction(CustomPreFunction.Remote("https://my.example.custom/function")))
      .BuildImageTag("logo.jpg")
Android:
MediaManager.get().url().transformation(new Transformation()
      .signed(true)
      .customFunction(new CustomPreFunction()
        .functionType("remote")
        .source("https://my.example.custom/function"))
      .generate("logo.jpg");
iOS:
imageView.cldSetImage(cloudinary.createUrl()
      .setTransformation(CLDTransformation()
        .setCustomPreFunction(CLDCustomFunction()
          .setFunctionType("remote")
          .setSource("https://my.example.custom/function")))
      .generate("logo.jpg", signUrl: true)!, cloudinary: cloudinary)

The code generates a URL similar to:

https://res.cloudinary.com/demo/image/upload/s--511a3ca4--/fn_pre:remote:aHR0cHM6Ly9teS5leGFtcGxlLmN1c3RvbS9mdW5jdGlvbg==//logo.jpg

A Preprocessing Example

The following example uses variables together with a preprocessing remote function in order to render a text string within an SVG file. The variables set the colors for the text (fillc) and the font stroke (strokec), as well as the text itself (msgstr). Note that the uploaded SVG file has been prepared to include variables, each of them specified with triple hash marks before and after the variable name (e.g., ###fillc###). The custom preprocessing remote function searches for these variables and replaces them with the given values.

Exhibit 1 - The SVG file uploaded to Cloudinary including the variable definitions:

<svg viewBox="0 0 1000 300"
  xmlns="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink">
  <defs>
    <path id="MyPath"
      d="M 100 200
      C 200 100 300   0 400 100
      C 500 200 600 300 700 200
      C 800 100 900 100 900 100" />
  </defs>
  <text x="10" y="100" font-size="40" fill="###fillc###" stroke="###strokec###">
    <textPath xlink:href="#MyPath">
       ###msgstr###
    </textPath>
  </text>
</svg>

Exhibit 2 - The SDK code for generating the image tag:

Ruby:
cl_image_tag("msg_tmp.svg",
      :sign_url => true,
      :variables => [
        ["$fillc", "!yellow!"],
        ["$strokec", "!black!"],
        ["$msgstr", "!The quick brown fox jumps over the lazy dog!"]],
      :custom_pre_function => {
        :function_type => "remote", 
        :source => "https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test"})
PHP:
cl_image_tag("msg_tmp.svg", array(
      "sign_url" => true,
      "variables" => array(
        ("$fillc", "!yellow!"),
        ("$strokec", "!black!"),
        ("$msgstr", "!The quick brown fox jumps over the lazy dog!")),
      "custom_pre_function" => array(
        "function_type" => "remote", 
        "source" => "https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test")))
Python:
CloudinaryImage("msg_tmp.svg").image(
      sign_url = True,
      variables = {
        "$fillc", "!yellow!",
        "$strokec", "!black!",
        "$msgstr", "!The quick brown fox jumps over the lazy dog!"},
      custom_pre_function = {
        "function_type" = "remote", 
        "source" = "https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test"})
Node.js:
cloudinary.image("msg_tmp.svg", {
      sign_url: true,
      variables => [
        ["$fillc", "!yellow!"],
        ["$strokec", "!black!"],
        ["$msgstr", "!The quick brown fox jumps over the lazy dog!"]],
      custom_pre_function:{
        function_type: "remote", 
        source: "https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test"}})
Java:
cloudinary.url().transformation(new Transformation()
      .signed(true)
      .variables(
        variable("$fillc", "!yellow!"),
        variable("$strokec", "!black!"),
        variable("$msgstr", "!The quick brown fox jumps over the lazy dog!")),
      .customPreFunction(
        remote("https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test")))
      .imageTag("msg_tmp.svg");
JS:
cloudinary.imageTag('msg_tmp.svg', {
      signUrl: true,
      variables: [
        ["$fillc", "!yellow!"],
        ["$strokec", "!black!"],
        ["$msgstr", "!The quick brown fox jumps over the lazy dog!"]],
      customPreFunction: new cloudinary.CustomFunction()
        .functionType("remote")
        .source("https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test")}).toHtml();
jQuery:
$.cloudinary.image('msg_tmp.svg', {
      signUrl: true,
      variables: [
        ["$fillc", "!yellow!"],
        ["$strokec", "!black!"],
        ["$msgstr", "!The quick brown fox jumps over the lazy dog!"]],
      customPreFunction: new cloudinary.CustomFunction()
        .functionType("remote")
        .source("https://urmai0up1j.execute-api.us-east-1.amazonaws.com/test")})

Exhibit 3 - The code generates a URL similar to:

http://res.cloudinary.com/demo/image/upload/s--kkSWOzr4--/$fillc_!yellow!,$strokec_!black!,$msgstr_!The quick brown fox jumps over the lazy dog!,fn_pre:remote:aHR0cHM6Ly91cm1haTB1cDFqLmV4ZWN1dGUtYXBpLnVzLWVhc3QtMS5hbWF6b25hd3MuY29tL3Rlc3Q=/msg_tmp.svg

Exhibit 4 - The resulting image:

Fully Extensible Image Processing

Cloudinary strives to enhance our core features as best we can, prioritizing features that will benefit many customers, and to be as responsive as possible to customer requests. As extensive as our open platform functionality may be, we may not currently support a specific functionality, and so custom functions give our customers the flexibility to incorporate their own functionality in the image-processing pipeline. It also makes it relatively simple to incorporate other 3rd party functionality. Make sure to read up on all the details about custom functions in the documentation, and note that custom functions are available with no extra charge for all Cloudinary's plans, including the free plan.

Recent Blog Posts

Converting Videos to GIFs in Kotlin-Based Android Apps

Among the image formats available today, Graphics Interchange Format (GIF) works for both static and animated images. A popular use of GIF images—commonly called GIFs—is to repeatedly loop motions, which you cannot in real life. Many people have turned themselves into online celebrities through effective application of GIFs.

Read more
Simple Steps to Make Your Site Accessible With Cloudinary

Web accessibility (a11y) is about gaining an insight into—

It’s a blurry line between accessibility and usability (or user experience) and one might argue that they are actually the same thing. Nonetheless, if your audience can't use your app with ease and confidence, then it’s not accessible enough and the user experience is not good enough.

Read more
Learn three ways in which to tell appealing stories on video.

Video storytelling is one of the most effective means of conveying messages to your audience. It’s also a popular marketing tool because most people are automatically drawn to video. In eCommerce, if accompanied by attractive images and a compelling narrative, video storytelling uniquely connect with potential customers without fail.

Read more