Posts

Showing posts from December, 2019

Difference between docker bind mount and docker volumes

One of the major design challenge you face while running your applications and services on docker containers is how to persist data. By default the docker containers do not store persistent data. Data is linked to the lifecycle of the container. To overcome the above challenge, persist data and de-link it from the lifecycle of the container docker provides two options 1. Bind Mounts  2. Volumes In this blog post I would be discussing in detail the above two methods to persist data while using containers with the pros and cons of using each approach. Bind Mounts Volumes A file/folder stored anywhere on the container host filesystem, mounted on a running container Preferred way to store persistent data Can exist anywhere on the filesystem Docker volumes exist at specific location within the filesystem. For Ubuntu they are at /var/lib/docker/volumes Processes outside of docker can also modify it Doc...

Data Science Series: Part1: Using Python, Numpy, Pandas, Camelot to extract AQI data for Delhi from reports published as PDF on CPCB website and generate line chart visualization

Preface : With the growing air pollution in India, it has become imperative to monitor the quality of air in different parts of the country to be able to check the volumes of pollution. There was need to standardize the metrics for air pollution hence the Central Pollution Control board in India came up with an Indices that factors in various pollutants in the air (PM2.5, PM10, O3, CO2, CO etc) and give a numeric value indicating the pollution level. CPCB on its website  https://cpcb.nic.in  publishes reports in PDF format that contains the AQI levels for a single day accross various cities in India. Objective: The Central Pollution 

Azure Cloud: Mounting Azure storage as a drive on windows machine

Image
One of the common challenges faced while working on the cloud is to use a consistent storage across multiple VM's to be able to retrieve data on all of them. Solution to this is to mount Azure File service (Part of Azure storage account) as a drive on your VM. In this blog I will walk you through the steps to achieve the same. First create a normal storage account on the Azure portal. Try to host the storage account in the same region as your VM to avoid latency issues during data transfer to and fro from the VM. Now you need to create a file share in that storage account which can only be done via either Azure Powershell or Azure API. In this blog we will be using Azure Powershell 1. Add-AzureRMAccount to configure powershell to use your Azure Account. 2.