# Application Insights

## Overview

**Application Insights is an Azure service used to collect logs. It is used in the pipeline to collect the logs of each service and allow users to track the status of jobs by viewing and querying logs.**

## **Storing Logs**

The first requirement is an Application Insights instance on Azure. This service generates a connection string which can be added to each application. In Python applications such as those in the validation pipeline, logs are pushed using a custom logger. By adding the connection string to the custom logger, the logs are automatically sent to Azure using the `AzureLogHandler` library.

```python
#setup logger
logging.basicConfig()
logger = logging.getLogger("azure-logger")
logger.setLevel(logging.INFO)

connect_str = os.getenv('APPLICATION_INSIGHTS_CONNECTION_STRING')
handler = AzureLogHandler(connection_string=connect_str)
handler.setLevel(logging.INFO)
logger.addHandler(handler)
```

## **Viewing Logs**

Once logs are sent to Azure, the next step is the ability to view them. Each Application Insights instance can be viewed in Azure. There are many functionalities, but the most important here are the logs, and more specifically the traces category.

![](https://962937113-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mi_YhQ1n5sSjCqtqMa4%2Fuploads%2FlA3mr43dFGJR5fZBxgvL%2Fimage.png?alt=media\&token=a486c447-538b-411d-9965-bbb79eb61754)

Incoming logs are sorted under this category. This is an example of a log instance:

![](https://962937113-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mi_YhQ1n5sSjCqtqMa4%2Fuploads%2FeGBlCDkMIrhiUs7VtGLd%2Fimage.png?alt=media\&token=cb74cfaa-ef69-4edd-b682-85c0684be8c9)

Logs have an added field called `customDimensions`. These are used in the application to define custom fields. In doing this, it is possible to attach the job UUID to the log.&#x20;

```python
self.properties = {'custom_dimensions': {'UUID': container_name}}
self.logger.info('This is a log!', extra=self.properties)
```

Another possible use of this service is [Alerts](https://docs.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-overview). With alerts it is possible to set up automatic notifications when a service goes down or a job fails.

## **Querying Logs**

Based on the aforementioned `customDimensions`, it is possible to create a query to get all logs that contain that specific UUID:&#x20;

```sql
traces 
| extend uuid = tostring(customDimensions["UUID"])
| where uuid == "338de6f55b"
```

The above query gets all logs where the UUID matches "338de6f55b". In Azure, this gives the following result:

![](https://962937113-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mi_YhQ1n5sSjCqtqMa4%2Fuploads%2FMjbPhPwjCOannj82o1KA%2Fimage.png?alt=media\&token=6056f1cf-ee41-40f0-9763-6d28fbdf3073)
