Data Factory

Official Documentation

Service Description

Data processing in today's companies is marked by heterogeneous data storage (SQL, NoSQL, unstructured data, etc.) and processing components (databases, Big Data processors, etc.). Data in a company often passes through complex paths from generation or receipt of the data, through various data processing components, to storage or distribution of the data to various recipients. With Data Factory, local data such as that from SQL Server can be processed together with cloud-related data from Azure SQL Database, Blobs, and Tables. These data processing streams can be created, processed, and monitored through simple, highly available data pipelines. Data sources and data recipients can be defined, and the movement of the data in the company can be traced and monitored from a central location.

Getting Started

  1. Introduction to Azure Data Factory
    10/7/2015, Video, 0:59:31
  2. Orchestrating Data and Services with Azure Data Factory
    2/25/2016, Mva
  3. New capabilities for data integration in the cloud
    9/28/2017, Video, 1:35:27
  4. Cortana Intelligence Suite End-to-End
    1/16/2017, Mva
  5. Intro to Data Factory Deep Dive
    10/7/2015, Video, 0:31:34
  6. Building Hybrid Big Data Pipelines with Azure Data Factory
    10/7/2015, Video, 1:11:26



Latest Content

Subscribe to News about Data Factory

Title  
Blog
Blog
Video
Video
Video
Blog
Blog
Blog
Blog
Blog
Video
Blog
more...


Azure Documentation

1. Data Factory V2 Documentation
2. Overview
     2.1. Introduction to Data Factory
3. Quickstarts
     3.1. Create data factory
          3.1.1. PowerShell
          3.1.2. .NET
          3.1.3. Python
          3.1.4. REST
          3.1.5. Portal
4. Tutorials
     4.1. 1 - Deploy SSIS packages to Azure
     4.2. 2 - Copy data in cloud
     4.3. 3 - Copy on-premises data to cloud
     4.4. 4 - Copy data in bulk
     4.5. 5 - Copy data incrementally
     4.6. 6 - Transform data in cloud
     4.7. 7 - Transform data in virtual network
     4.8. 8 - Control flow
5. Samples
     5.1. Code samples
     5.2. PowerShell
6. Concepts
     6.1. Pipelines and activities
     6.2. Datasets and linked services
     6.3. Pipeline execution and triggers
     6.4. Integration runtime
7. How-to guides
     7.1. Move data
          7.1.1. Copy Activity overview
          7.1.2. Connectors
               7.1.2.1. Amazon Redshift
               7.1.2.2. Amazon S3
               7.1.2.3. Azure Blob Storage
               7.1.2.4. Azure Cosmos DB
               7.1.2.5. Azure Data Lake Store
               7.1.2.6. Azure Search
               7.1.2.7. Azure SQL Database
               7.1.2.8. Azure SQL Data Warehouse
               7.1.2.9. Azure Table Storage
               7.1.2.10. Cassandra
               7.1.2.11. DB2
               7.1.2.12. Dynamics 365
               7.1.2.13. Dynamics CRM
               7.1.2.14. File System
               7.1.2.15. FTP
               7.1.2.16. GE Historian
               7.1.2.17. HDFS
               7.1.2.18. HTTP
               7.1.2.19. Informix
               7.1.2.20. Microsoft Access
               7.1.2.21. MongoDB
               7.1.2.22. MySQL
               7.1.2.23. OData
               7.1.2.24. ODBC
               7.1.2.25. Oracle
               7.1.2.26. PostgreSQL
               7.1.2.27. Salesforce
               7.1.2.28. SAP Business Warehouse
               7.1.2.29. SAP HANA
               7.1.2.30. SFTP
               7.1.2.31. SQL Server
               7.1.2.32. Sybase
               7.1.2.33. Teradata
               7.1.2.34. Web Table
          7.1.3. Format and compression support
          7.1.4. Schema and type mapping
          7.1.5. Fault tolerance
          7.1.6. Performance and tuning
     7.2. Transform data
          7.2.1. HdInsight Hive Activity
          7.2.2. HdInsight Pig Activity
          7.2.3. HdInsight MapReduce Activity
          7.2.4. HdInsight Streaming Activity
          7.2.5. HdInsight Spark Activity
          7.2.6. ML Batch Execution Activity
          7.2.7. ML Update Resource Activity
          7.2.8. Stored Procedure Activity
          7.2.9. Data Lake U-SQL Activity
          7.2.10. Custom activity
          7.2.11. Compute linked services
     7.3. Control flow
          7.3.1. Web Activity
          7.3.2. Lookup Activity
          7.3.3. Get Metadata Activity
          7.3.4. Execute Pipeline Activity
          7.3.5. For Each Activity
          7.3.6. Expression Language
          7.3.7. System variables
     7.4. Security
          7.4.1. Data movement security considerations
          7.4.2. Store credentials in Azure Key Vault
          7.4.3. Encrypt credentials for self-hosted integration runtime
     7.5. Monitor and manage
          7.5.1. Use .NET
          7.5.2. Use Azure Monitor
          7.5.3. Monitor visually
          7.5.4. Manage Azure-SSIS integration runtime
          7.5.5. Monitor integration runtime
     7.6. Create integration runtime
          7.6.1. Azure integration runtime
          7.6.2. Self hosted integration runtime
          7.6.3. Azure-SSIS integration runtime
          7.6.4. Join Azure-SSIS integration runtime to a VNET
8. Reference
     8.1. .NET
     8.2. PowerShell
     8.3. REST API
     8.4. Python
9. Resources
     9.1. FAQ
     9.2. Azure Roadmap
     9.3. Pricing
     9.4. MSDN forum
     9.5. Stack Overflow
     9.6. Request a feature
     9.7. Region availability
     9.8. Support options

Online Training Content

Date Title
5/24/2017 Orchestrating Big Data with Azure Data Factory
1/16/2017 Cortana Intelligence Suite End-to-End
7/4/2016 Design and Implement Big Data & Advanced Analytics Solutions
2/25/2016 Orchestrating Data and Services with Azure Data Factory

Tools

Tool Description

Videos

Date Title Length
9/30/2017 Deep dive into SQL Server Integration Services (SSIS) 2017 and beyond 1:25:20
9/30/2017 Getting peak performance from your SQL Data Warehouse column store 1:15:23
9/28/2017 New capabilities for data integration in the cloud 1:35:27
4/2/2017 Cloud Tech 10 - 3rd April 2017 0:08:48
3/27/2017 Cloud Tech 10 - 27th March 2017 0:08:37
3/13/2017 Cloud Tech 10 - 13th March 2017 0:09:06
2/10/2017 Design for Big Data with Microsoft Azure SQL Data Warehouse 1:02:19
2/10/2017 Orchestrating Big Data Pipelines with Azure Data Factory 1:15:24
2/10/2017 Interview with Lace Lofranco 1:02:06
7/13/2016 Advancements in Data Technology 0:29:42

Page 1 of 4