Data Factory

Official Documentation

Service Description

Data processing in today's companies is marked by heterogeneous data storage (SQL, NoSQL, unstructured data, etc.) and processing components (databases, Big Data processors, etc.). Data in a company often passes through complex paths from generation or receipt of the data, through various data processing components, to storage or distribution of the data to various recipients. With Data Factory, local data such as that from SQL Server can be processed together with cloud-related data from Azure SQL Database, Blobs, and Tables. These data processing streams can be created, processed, and monitored through simple, highly available data pipelines. Data sources and data recipients can be defined, and the movement of the data in the company can be traced and monitored from a central location.

Getting Started

  1. Azure Data Factory Learning Path
    9/7/2016, Webpage
  2. Introduction to Azure Data Factory
    10/7/2015, Video, 0:59:31
  3. Orchestrating Data and Services with Azure Data Factory
    2/25/2016, Mva
  4. Cortana Intelligence Suite End-to-End
    1/16/2017, Mva
  5. Intro to Data Factory Deep Dive
    10/7/2015, Video, 0:31:34
  6. Building Hybrid Big Data Pipelines with Azure Data Factory
    10/7/2015, Video, 1:11:26

Latest Content

Subscribe to News about Data Factory


Azure Documentation

1. Overview
     1.1. Introduction to Azure Data Factory
     1.2. Concepts
          1.2.1. Pipelines and activities
          1.2.2. Datasets
          1.2.3. Scheduling and execution
2. Get Started
     2.1. Tutorial: Create a pipeline to copy data
          2.1.1. Copy Wizard
          2.1.2. Azure portal
          2.1.3. Visual Studio
          2.1.4. PowerShell
          2.1.5. Azure Resource Manager template
          2.1.6. REST API
          2.1.7. .NET API
     2.2. Tutorial: Create a pipeline to transform data
          2.2.1. Azure portal
          2.2.2. Visual Studio
          2.2.3. PowerShell
          2.2.4. Azure Resource Manager template
          2.2.5. REST API
     2.3. Tutorial: Move data between on-premises and cloud
     2.4. FAQ
3. How To
     3.1. Move Data
          3.1.1. Copy Activity Overview
          3.1.2. Data Factory Copy Wizard
      Load 1 TB in 15 minutes
          3.1.3. Performance and tuning guide
          3.1.4. Fault tolerance
          3.1.5. Security considerations
          3.1.6. Connectors
      Amazon Redshift
      Amazon S3
      Azure Blob Storage
      Azure Cosmos DB
      Azure Data Lake Store
      Azure Search
      Azure SQL Database
      Azure SQL Data Warehouse
      Azure Table Storage
      File system
      SAP Business Warehouse
      SAP HANA
      SQL Server
      Web table
          3.1.7. Data Management Gateway
      High availability and scalability (Preview)
      Walkthrough: Move data from an on-premises SQL Server to Azure Blob Storage
     3.2. Transform Data
          3.2.1. HDInsight Hive Activity
          3.2.2. HDInsight Pig Activity
          3.2.3. HDInsight MapReduce Activity
          3.2.4. HDInsight Streaming Activity
          3.2.5. HDInsight Spark Activity
          3.2.6. Machine Learning Batch Execution Activity
          3.2.7. Machine Learning Update Resource Activity
          3.2.8. Stored Procedure Activity
          3.2.9. Data Lake Analytics U-SQL Activity
          3.2.10. .NET custom activity
          3.2.11. Invoke R scripts
          3.2.12. Reprocess models in Azure Analysis Services
          3.2.13. Compute Linked Services
     3.3. Develop
          3.3.1. Azure Resource Manager template
          3.3.2. Samples
          3.3.3. Functions and system variables
          3.3.4. Naming rules
          3.3.5. .NET API change log
     3.4. Monitor and Manage
          3.4.1. Monitoring and Management app
          3.4.2. Azure Data Factory pipelines
          3.4.3. Using .NET SDK
          3.4.4. Troubleshoot Data Factory issues
          3.4.5. Troubleshoot issues with using Data Management Gateway
4. Reference
     4.1. Code samples
     4.2. PowerShell
     4.3. .NET
     4.4. REST
     4.5. JSON
5. Resources
     5.1. Azure Roadmap
     5.2. Case Studies
     5.3. Learning path
     5.4. MSDN Forum
     5.5. Pricing
     5.6. Pricing calculator
     5.7. Release notes for Data Management Gateway
     5.8. Request a feature
     5.9. Service updates
     5.10. Stack Overflow
     5.11. Videos
          5.11.1. Customer Profiling
          5.11.2. Process large-scale datasets using Data Factory and Batch
          5.11.3. Product Recommendations

Web Content

Content Type
Azure Data Factory Learning Path Webpage

Online Training Content

Date Title
5/24/2017 Orchestrating Big Data with Azure Data Factory
1/16/2017 Cortana Intelligence Suite End-to-End
7/4/2016 Design and Implement Big Data & Advanced Analytics Solutions
2/25/2016 Orchestrating Data and Services with Azure Data Factory


Tool Description


Date Title Length
4/2/2017 Cloud Tech 10 - 3rd April 2017 0:08:48
3/27/2017 Cloud Tech 10 - 27th March 2017 0:08:37
2/10/2017 Design for Big Data with Microsoft Azure SQL Data Warehouse 1:02:19
2/10/2017 Orchestrating Big Data Pipelines with Azure Data Factory 1:15:24
2/10/2017 Interview with Lace Lofranco 1:02:06
7/13/2016 Advancements in Data Technology 0:29:42
7/12/2016 Hadoop without borders – learn to build hybrid big data analytics pipelines with Azure Data Factory 0:58:50
6/29/2016 Azure Data Factory - Monitoring and Managing Big Data Pipelines 0:11:54
6/7/2016 Azure Data Factory - Monitoring and Managing Big Data Piplines 0:11:53
5/5/2016 Data Factory Concepts 0:06:13

Page 1 of 3