- Download 2
- File Size 19.01 MB
- File Count 1
- Create Date October 11, 2020
- Last Updated May 23, 2021
Azure Tutorial | Azure DevOps | Understanding Azure Data Factory
What Is Azure Data Factory?
Big Data requires a service that can help you orchestrate and operationalize complex processes that in turn refine the enormous structure/semi-structured data into actionable business insights.
Azure Data Factory (ADF) is a cloud-based data integration service that acts as the glue in your Big Data or advanced analytics solution, ensuring your complex workflows integrate with the various dependent services required in your solution. It provides a single pane for monitoring
all your data movements and complex data processing jobs. Simply said, it is serverless, managed cloud service that’s built for these complex hybrid ETL, ELT, and data integration projects (data integration as a service).
Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores. It can process and transform the data by using compute services such as Azure HDInsight Hadoop, Spark, Azure Data Lake Analytics, and Azure Machine Learning
Azure Data Factory is the de facto tool for building end-to-end advanced analytics solutions on Azure. It can handle complex ETL data workflows and integrates natively with all Azure services with enterprise-grade security offerings.
For ease of authoring and to make you more productive, it offers a drag-and-drop user interface with rich control flow for building complex data workflows, and it provides a single-pane-of-glass monitoring solution for your data pipelines.
Something that really stands out is the low price-to-performance ratio, being cost-effective and performant at the same time. Its data movement capabilities with more than 75 high-performance connectors are extremely helpful when dealing with Big Data coming from various sources. To give you an example, the 100GB data movement would cost you less than $0.40 (that is correct, 40 cents). ADF is an Azure service and bills you in a pay-as-you-go model against your Azure subscription with no up-front costs.
Azure Data Factory (ADF) also supports operationalizing existing SSIS packages on the cloud, which is helpful if you are modernizing your data warehouse solution over time with a lot of existing SSIS packages.