Data Factory

Official Documentation

Service Description

Data processing in today's companies is marked by heterogeneous data storage (SQL, NoSQL, unstructured data, etc.) and processing components (databases, Big Data processors, etc.). Data in a company often passes through complex paths from generation or receipt of the data, through various data processing components, to storage or distribution of the data to various recipients. With Data Factory, local data such as that from SQL Server can be processed together with cloud-related data from Azure SQL Database, Blobs, and Tables. These data processing streams can be created, processed, and monitored through simple, highly available data pipelines. Data sources and data recipients can be defined, and the movement of the data in the company can be traced and monitored from a central location.

Getting Started

  1. Azure Data Factory Learning Path
    9/7/2016, Webpage
  2. Introduction to Azure Data Factory
    10/7/2015, Video, 0:59:31
  3. Orchestrating Data and Services with Azure Data Factory
    2/25/2016, Mva
  4. Cortana Intelligence Suite End-to-End
    1/16/2017, Mva
  5. Intro to Data Factory Deep Dive
    10/7/2015, Video, 0:31:34
  6. Building Hybrid Big Data Pipelines with Azure Data Factory
    10/7/2015, Video, 1:11:26

Latest Content

RSS Feed

Title  
Azure Data Factory February new features update Blog
Data Factory supports multiple web service inputs for Azure ML Batch Execution Blog
Ingest data from Apache Cassandra, Salesforce and Data Management Gateway 2.0 release Blog
Advancements in Data Technology Video
Hadoop without borders – learn to build hybrid big data analytics pipelines with Azure Data Factory Video
Azure Data Factory - Monitoring and Managing Big Data Pipelines Video
Azure Data Factory - Monitoring and Managing Big Data Piplines Video
Data movement made faster in Azure Blog
Data Factory Concepts Video
Data Integration in the Cloud and Building Data Analytics Pipelines Video
Simple and reliable data movement with Azure Data Factory Copy Wizard Blog
Code-free copy wizard for Azure Data Factory Blog

Azure Documentation

1. Overview
     1.1. Introduction to Azure Data Factory
     1.2. Concepts
          1.2.1. Datasets
          1.2.2. Pipelines and activities
          1.2.3. Scheduling and execution
          1.2.4. Compute Linked Services
2. Get Started
     2.1. Tutorial: Create a pipeline to copy data
          2.1.1. Copy Wizard
          2.1.2. Azure portal
          2.1.3. Visual Studio
          2.1.4. PowerShell
          2.1.5. Azure Resource Manager template
          2.1.6. REST API
          2.1.7. .NET API
     2.2. Tutorial: Create a pipeline to transform data
          2.2.1. Azure portal
          2.2.2. Visual Studio
          2.2.3. PowerShell
          2.2.4. Azure Resource Manager template
          2.2.5. REST API
     2.3. FAQ
3. How To
     3.1. Move Data
          3.1.1. Data Factory Copy Wizard
               3.1.1.1. Load 1 TB in 15 minutes
          3.1.2. Copy Activity
               3.1.2.1. Performance & Tuning Guide
               3.1.2.2. Amazon Redshift
               3.1.2.3. Amazon S3
               3.1.2.4. Azure Blob Storage
               3.1.2.5. Azure Data Lake Store
               3.1.2.6. Azure DocumentDB
               3.1.2.7. Azure Search
               3.1.2.8. Azure SQL Database
               3.1.2.9. Azure SQL Data Warehouse
               3.1.2.10. Azure Table Storage
               3.1.2.11. Cassandra
               3.1.2.12. DB2
               3.1.2.13. File system
               3.1.2.14. FTP
               3.1.2.15. HDFS
               3.1.2.16. HTTP
               3.1.2.17. MongoDB
               3.1.2.18. MySQL
               3.1.2.19. OData
               3.1.2.20. ODBC
               3.1.2.21. Oracle
               3.1.2.22. PostgreSQL
               3.1.2.23. Salesforce
               3.1.2.24. SAP Business Warehouse
               3.1.2.25. SFTP
               3.1.2.26. SAP HANA
               3.1.2.27. SQL Server
               3.1.2.28. Sybase
               3.1.2.29. Teradata
               3.1.2.30. Web table
          3.1.3. Data Management Gateway
          3.1.4. Move data between on-premises and cloud
          3.1.5. Map input and output dataset columns
          3.1.6. Supported file and compression formats
     3.2. Transform Data
          3.2.1. HDInsight Hive Activity
          3.2.2. HDInsight Pig Activity
          3.2.3. HDInsight MapReduce Activity
          3.2.4. HDInsight Streaming Activity
          3.2.5. HDInsight Spark Activity
          3.2.6. Machine Learning Batch Execution Activity
          3.2.7. Machine Learning Update Resource Activity
          3.2.8. Stored Procedure Activity
          3.2.9. Data Lake Analytics U-SQL Activity
          3.2.10. .NET custom activity
          3.2.11. Invoke R scripts
     3.3. Develop
          3.3.1. Azure Resource Manager template
          3.3.2. Samples
          3.3.3. Functions and system variables
          3.3.4. Naming rules
          3.3.5. .NET API change log
     3.4. Monitor and Manage
          3.4.1. Monitoring and Management app
          3.4.2. Azure Data Factory pipelines
          3.4.3. Using .NET SDK
          3.4.4. Troubleshoot Data Factory issues
          3.4.5. Troubleshoot issues with using Data Management Gateway
4. Reference
     4.1. PowerShell
     4.2. .NET
     4.3. REST
5. Resources
     5.1. Release notes for Data Management Gateway
     5.2. Learning path
     5.3. Case Studies
          5.3.1. Product Recommendations
          5.3.2. Customer Profiling
          5.3.3. Process large-scale datasets using Data Factory and Batch
     5.4. Service updates
     5.5. Pricing
     5.6. MSDN Forum
     5.7. Stack Overflow
     5.8. Videos
     5.9. Request a feature

Web Content

Content Type
Azure Data Factory Learning Path Webpage

Tools

Tool Description

Videos

Date Title Length
7/13/2016 Advancements in Data Technology 0:29:42
7/12/2016 Hadoop without borders – learn to build hybrid big data analytics pipelines with Azure Data Factory 0:58:50
6/29/2016 Azure Data Factory - Monitoring and Managing Big Data Pipelines 0:11:54
6/7/2016 Azure Data Factory - Monitoring and Managing Big Data Piplines 0:11:53
5/5/2016 Data Factory Concepts 0:06:13
3/30/2016 Data Integration in the Cloud and Building Data Analytics Pipelines 0:33:03
10/7/2015 Building Hybrid Big Data Pipelines with Azure Data Factory 1:11:26
10/7/2015 Intro to Data Factory Deep Dive 0:31:34
10/7/2015 Introduction to Azure Data Factory 0:59:31
9/17/2015 Operationalizing Your End-to-End Analytical Solution 0:56:12

Page 1 of 2