Azure Functions

How Azure functions are used within the application.

Overview

Azure Functions are triggered when uploading a file to the BLOB storage, triggering the next service in the pipeline.

The Purpose of Cloud Functions

Cloud functions are serverless functions that are triggered based on events set up by the developer. Take the example of a BLOB storage and a database. The BLOB storage contains images, while the database contains references to these images. If, for some reason, an image was deleted from the BLOB storage, the associated reference to that image (which now references nothing!) would still be present in the database. This could be removed manually, but you could instead set up a function that does this automatically.

In the Context of the Avatar Validation Pipeline

The pipeline uses cloud functions (called Function Apps in Azure) to trigger the next service in the pipeline. A schematic overview can be seen here.

Each time something is uploaded to the BLOB storage, the blob-upload function app is triggered. Based on the file type of what is uploaded, one of two services can be triggered: the OpenFace analysis in the case of an image/video, or the Analysis module in the case of a .csv file (indicating the OpenFace analysis has already been completed).

def main(event: func.EventGridEvent):
    result = json.dumps({
        'id': event.id,
        'data': event.get_json(),
        'topic': event.topic,
        'subject': event.subject,
        'event_type': event.event_type,
    })

    container_name = event.subject.split('/')[4]
    file_name = event.subject.split('/')[6]

    logging.info('Python EventGrid trigger processed an event: %s', result)
    logging.info("Function triggered concerning container: %s and file: %s", container_name, file_name)

    if len(container_name) == 10: 
        if '.csv' in file_name:
            logging.info('Received .csv file.')
            trimmed = file_name.replace('.csv', '')
            parts = trimmed.split('_')
            logging.info("Building URL: http://avatar-analysis-service.westeurope.azurecontainer.io:5000/container/" + container_name + '/' + parts[-2] + '/' + parts[-1])
            try:
                r = requests.get(url="http://avatar-analysis-service.westeurope.azurecontainer.io:5000/container/" + container_name + '/' + parts[-2] + '/' + parts[-1])
                logging.info("Called Analysis service with previous URL. Status code: %s", r.status_code)
            except Exception as e:
                logging.critical(e)
                logging.info("Could not perform call to analysis service.")
        if '.png' in file_name or '.jpg' in file_name or '.jpeg' in file_name:
            logging.info('Received image.')
            logging.info("Building URL: http://openface-rest-service.westeurope.azurecontainer.io:5000/analyse/photo/IW/%s", container_name)
            try:
                r = requests.get(url="http://openface-rest-service.westeurope.azurecontainer.io:5000/analyse/photo/IW/" + container_name)
                logging.info("Called OpenFace service with previous URL. Status code: %s", r.status_code)
            except Exception as e:
                logging.critical(e)
                logging.info("Could not perform call to OpenFace service.")
        else:
            logging.critical('Unsupported file received')

Logic App Setup

There are two ways to set up the trigger for the aforementioned function: a BLOB-storage specific trigger or making use of EventGrid. A comparison of the two can be found here. The main difference in this context is that the BLOB specific trigger can have far more latency if it has a cold start. This could cause interruptions in the pipeline, causing me to use EventGrid instead.