Documentation

Homepage

Service Description

Data processing in today's companies is marked by heterogeneous data storage (SQL, NoSQL, unstructured data, etc.) and processing components (databases, Big Data processors, etc.). Data in a company often passes through complex paths from generation or receipt of the data, through various data processing components, to storage or distribution of the data to various recipients. With Data Factory, local data such as that from SQL Server can be processed together with cloud-related data from Azure SQL Database, Blobs, and Tables. These data processing streams can be created, processed, and monitored through simple, highly available data pipelines. Data sources and data recipients can be defined, and the movement of the data in the company can be traced and monitored from a central location.

Pricing SLAs Limits Learning Twitter

Getting Started

  1. 10/7/2015, Video, 0:59:31
    The data landscape is more varied than ever with unstructured and structured data originating from many cloud and on-premises sources. Data Factory enables you to process...
  2. 2/25/2016, Trainings
    Exploring data orchestration concepts? Check out this course on the basic capabilities of Azure Data Factory (ADF). Get an overview of advanced analytics, and see how Azure...
  3. 1/16/2017, Trainings
    If you’d like to learn how to architect solutions in Cortana Intelligence Suite and how to build intelligence into your applications, don’t miss this workshop! Build an...
  4. 10/7/2015, Video, 0:31:34
    This video takes a deeper dive into the features and functions of the Azure Data Factory orchestration engineLearn more: http://aka.ms/g7hlpt
  5. 10/7/2015, Video, 1:11:26
    This video takes a deeper dive into how Azure Data Factory can be used to build hybrid big data analytics pipelines through the lens of an automated risk processing use case...

Data Factory News

1/18/2020, Service Updates
Azure Synapse Analytics introduced a new COPY statement (preview) which provides the most flexibility for high-throughput data ingestion.
1/18/2020, Service Updates
Azure Data Factory copy activity now supports resume from last failed run when you copy files between file-based data stores to ease large volumes of data ingestion and migration.
1/18/2020, Service Updates
Azure Data Factory now supports SFTP as a sink and as a source. Use copy activity to copy data from any supported data store to your SFTP server located on-premises or in the cloud.
1/16/2020, YouTube: Microsoft Ignite [0:48:59]
Start by creating data flows in debug mode so that you can validate your transformation logic interactively. Next, add a data flow activity to your pipeline to execute and test your data flow in...
1/14/2020, YouTube: Microsoft Ignite [0:46:31]
In this experience, walk through creating a pipeline copy activity to copy a file to an Azure blob storage container, so we can prepare the file to be processed later for transformation.
1/14/2020, YouTube: Microsoft Ignite [0:46:28]
In this experience, see how Azure Data Factory (ADF), Azure Databricks, and Azure SQL Data Warehouse (SQL DW) can be used together to build a modern data warehouse. Start by using Azure Data...
1/14/2020, YouTube: Microsoft Ignite [0:47:40]
This session focuses on the all PaaS solution of Azure SQL DB/Managed Instance (MI) + SSIS in Azure Data Factory (ADF) to lift and shift, modernize, and extend ETL workflows. We first show you...
1/14/2020, YouTube: Microsoft Ignite [0:41:31]
Customers face common data integration problems every day. This session showcases how to use Azure SQL Data Warehouse, Azure Data Factory, Azure Data Lake Store, Azure Databricks, Azure Key Vault...
1/14/2020, YouTube: Microsoft Ignite [0:57:52]
In this session, we talk about the common data ingest challenges, patterns, and best practices based on our learnings from working directly with customers and partners. We walk through real-world...
1/14/2020, YouTube: Microsoft Ignite [0:19:50]
This session explores how Microsoft Cloud services can help transform analytics data into intelligent information. Throughout this session, explore building a smart city solution with Azure IoT...
1/14/2020, YouTube: Microsoft Ignite [0:20:50]
Azure Data Factory pricing is easy, right? No upfront costs. Pay only for what you use. It's a wonderful world for developers. Just a few clicks and your solution is ready for production! But...
1/14/2020, YouTube: Microsoft Ignite [0:44:44]
You already know how to build, orchestrate, and monitor data pipelines in Azure Data Factory (ADF). But how do you go from basic, hardcoded data pipelines to making your solution dynamic and...
12/16/2019, Service Updates
Azure Data Factory expands parameterization capabilities to make data flows even more reusable and scalable.
12/12/2019, Service Updates
Azure Data Factory copy activity now supports preserving metadata during file copy among Amazon S3, Azure Blob, and Azure Data Lake Storage Gen2.
11/11/2019, Service Updates
Azure Data Factory has added native CosmosDB and Avro connectors to mapping data flows
11/5/2019, Ignite 2019 [0:45:00]
Start by creating data flows in debug mode so that you can validate your transformation logic interactively. Next, add a data flow activity to your pipeline to execute and test your data flow in...
11/5/2019, Ignite 2019 [0:45:00]
Tailwind Traders deals with gigantic amounts of data, continually pouring in from sources such as sales, marketing, and fulfillment. In the past, the only choice they had was to put it all into a...
11/5/2019, Ignite 2019 [1:15:00]
This session focuses on the new features to unblock/accelerate cloud migration of legacy SQL Server Integration Services (SSIS), such as 2008 or earlier versions, with Azure Data Factory (ADF)....
11/5/2019, Ignite 2019 [0:45:00]
In this experience, walk through creating a pipeline copy activity to copy a file to an Azure blob storage container, so we can prepare the file to be processed later for transformation.
11/5/2019, Ignite 2019 [0:45:00]
This session focuses on the all PaaS solution of Azure SQL DB/Managed Instance (MI) + SSIS in Azure Data Factory (ADF) to lift and shift, modernize, and extend ETL workflows. We first show you...