Large-Scale Image Augmentation Techniques in Python

Images serve as crucial data for machine learning (ML) models, much like versatile tools in a craftsman’s kit. Yet, they often require tailored enhancements to meet the many demands of applications. Enter image augmentation — the process that equips these models with adaptability and resilience across diverse scenarios.

Augmentation broadens the model’s learning spectrum. Consider training a facial expression recognition model that predominantly sees left-sided expressions. Now, if you flip those images, the machine suddenly gains exposure to right-sided expressions without needing entirely new images. Image augmentation bridges these gaps, providing a holistic view for comprehensive learning.

Consider another scenario. Your machine needs to recognize objects under various lighting conditions. By automatically transforming images to vary their brightness and darkness, you can teach the model without investing in costly photo shoots. 

In this article, we’ll introduce large-scale augmentation techniques in Python that not only cover your basic image augmentation needs, but also provide extended capabilities to enable you to apply precise, AI-powered augmentation to your images.

You might already use Python libraries for image augmentation, but have you explored Cloudinary?

Cloudinary’s Python SDK offers:

• Familiar techniques. Leverage the techniques you’ve always used, like horizontal flipping to diversify angle recognition capabilities, brightness adjustment for flexibility with different lighting conditions, blur or sharpen effects for clarity variability, and rotation for adjustment to different orientations.

• Tailored augmentation. Automatically customize your augmentations to specific images using Cloudinary’s object detection and intuitive AI-detected image characteristics.

• Generative AI. Empowering beyond convention with object removal, outpainting, and replacement. These unparalleled capabilities empower more effective machine learning.

All supported by Cloudinary’s infrastructure. Offload images for large-scale training, preventing slow processing or failure, regardless of dataset size.

Here’s what you need to do to use Cloudinary for augmenting your datasets: 

• Sign up for a Cloudinary account.

• Install the Cloudinary Python library using pip install cloudinary.

• Find your credentials in your Dashboard.

• Follow along with our sample Jupyter Notebook.

OR

• Use your own images in your Python environment:
  • Configure Cloudinary in your settings.py with your credentials, ensuring they remain secure, then loop through your dataset and upload the images using the upload method of the Upload API. Save the public IDs (unique identifier for the image in Cloudinary) and delivery URLs from the responses in an array.

# Configure Cloudinary in your settings.py with your credentials, ensuring they remain secure.
# settings.py
import cloudinary
import cloudinary.uploader
import cloudinary.api
from cloudinary.utils import cloudinary_url
import cloudinary.api	
cloudinary.config( 
  	cloud_name = "your_cloud_name",
  	api_key = "your_api_key",
  	api_secret = "your_api_secret"
)

# The upload statement for a single media asset.
response = cloudinary.uploader.upload(file, use_filename=true)
images.append({'pubicId': response.public_id,'url': response.secure_url})Code language: PHP (php)

Remember when we talked about tailored enhancements for specific images? Here’s an example. Let’s say you’re aiming to augment only images with faces — a task made easy with Cloudinary’s object detection. Tailored augmentation becomes a straightforward process, enhancing images based on their distinct features.

# Code snippet for object detection-based augmentation 
details = cloudinary.api.resource("<public_id>", faces=True)
facecount = len(details['faces'])

if facecount > 0:
    flipped_url, options = cloudinary.utils.cloudinary_url("<public_id>", transformation=[
        {'angle': 'hflip'}
    ])
    # Use 'flipped_url' for further processing or operations
Code language: PHP (php)

You’re not limited to face detection. Cloudinary’s powerful image analysis and object detection capabilities allow you to choose which images to augment based on a wide variety of characteristics. For more information, see Coudinary AI Content Analysis.

Cloudinary effectively removes and replaces unwanted elements in images, generating new training samples. This process enhances the model’s adaptability in interpreting scenes. For instance, removing obstructing parked cars from street scene images refines the model’s understanding of diverse real-world scenarios.

You can specify one or more objects that you want to remove:

python
# Code snippet for object removal 
object_removed_url, options = cloudinary.utils.cloudinary_url("<public_id>", transformation=[
    {'effect': 'remove:prompt:parked_car', 'multiple': True}
])
# Use 'object_removed_url' for further processing or operations
Code language: PHP (php)

Outpainting, a technique powered by Cloudinary, broadens an image’s perspective. It extends the canvas around focused objects, like birds, enhancing the model’s accuracy in detecting them amidst varied backgrounds.

python
# Code snippet for generative fill 
outpaint_url, options = cloudinary.utils.cloudinary_url("docs/bird.jpg", 
    transformation=[
        {'aspect_ratio': "16:9", 'background': "gen_fill", 'width': 1500, 'crop': "pad"}
    ]
)
# Use 'outpaint_url' for further processing or operations
Code language: PHP (php)

These code snippets demonstrate Cloudinary’s transformation capabilities for various basic augmentations, from adjusting lighting to applying blur/sharpen effects and rotating images at different angles.

Replace <public_id> with the public ID of your image. 

# Adjust the lighting to evening 
evening_url, options = cloudinary.utils.cloudinary_url('<public_id>', transformation=[
   {'effect': "art:frost"}
])
# Use 'evening_url' for further processing or operations

# Adjust lighting to morning
morning_url, options = cloudinary.utils.cloudinary_url('<public_id>', transformation=[
   {'effect': "art:sizzle"}
])
# Use 'morning_url' for further processing or operations
Code language: PHP (php)

# Blur
blurred_url, options = cloudinary.utils.cloudinary_url('<public_id>', transformation=[
    {'effect': 'blur:300'}
])
# Use 'blurred_url' for further processing or operations


#Sharpen
blurred_url, options = cloudinary.utils.cloudinary_url('<public_id>', transformation=[
    {'effect': 'sharpen:100'}
])
# Use 'blurred_url' for further processing or operations
Code language: PHP (php)

# Rotate
rotated_url, options = cloudinary.utils.cloudinary_url('<public_id>', transformation=[
    {'angle': 20}
])
# Use 'rotated_url' for further processing or operations
Code language: PHP (php)

Image augmentation is essential for training robust computer vision models, enabling them to learn from a rich tapestry of image variations. With Cloudinary’s Python SDK, equipped with powerful infrastructure, cutting-edge AI techniques and the ability to handle images at an immense scale, training becomes streamlined, empowering models to grasp the intricacies of complex images. 

Ready to elevate your machine learning capabilities? Explore large-scale augmentation techniques in Python and revolutionize your model training today.

If you found this article helpful and want to discuss it in more detail, head over to Cloudinary Community forum and its associated Discord.