Getting Data from AWS data lake into Azure Data Lake
With Organizations becoming more cloud agnostic time and again a need arises to migrate your data from one cloud platform to another to make use of the ETL and Machine Learning platform on another cloud vendor. In this article we would be covering how to migrate your data from AWS S3 data Lake to Azure Data Lake storage account. We would be using az copy for the purpose. Step1. Authenticate to AWS account where the AWS data lake resides in. For this purpose we would be using the User security Access key and ID set AWS_ACCESS_KEY_ID=<value> set AWS_SECRET_ACCESS_KEY=<value> Step2: Authenticate to the Azure Data Lake storage account. For this purpose we would be using the security Access tokens generated from the storage account where the data lake resides in and embed the same in the URL of the container where we want to put our data in Step3- Initiate the data transfer via Az Copy azcopy copy "https://azuremstnks.s3.ap-south-1.amazonaws.com/weather/" "https...