Overview
This is a 6-week evening program providing a hands-on introduction to the Hadoop and Spark ecosystem of Big Data technologies. The course will cover these key components of Apache Hadoop: HDFS, MapReduce with streaming, Hive, and Spark. Programming will be done in Python. The course will begin with a review of Python concepts needed for our examples. The course format is interactive. Students will need to bring laptops to class. We will do our work on AWS (Amazon Web Services); instructions will be provided ahead of time on how to connect to AWS and obtain an account.
What is Hadoop?
Hadoop is a set of open-source programs running in computer clusters that simplify the handling of large amounts of data. Originally, Hadoop consisted of a distributed file system tuned for large data sets and an implementation of the MapReduce parallelism paradigm, but has expanded in many ways. It now includes database systems, languages for parallelism, libraries for machine learning, its own job scheduler, and much more. Furthermore, MapReduce is no longer the only parallelism framework; Spark is an increasingly popular alternative. In summary, Hadoop is a very popular and rapidly growing set of cluster computing solutions, which is becoming an essential tool for data scientists.
Syllabus
Unit 1 – Introduction: Hadoop, MapReduce, Python
Overview of Big Data and the Hadoop ecosystem
The concept of MapReduce
HDFS – Hadoop Distributed File System
Python for MapReduce
Unit 2 – MapReduce
More Python for MapReduce
Implementing MapReduce with Python streaming
Unit 3 – Hive: A database for Big Data
Hive concepts, Hive query language (HiveQL)
User-defined functions in Python (using streaming)
Accessing Hive from Python
Unit 4 – Pig: A Platform for Analyzing Large Datasets Using MapReduce
Intro to Apache Pig
Data Types in Pig
Pig Latin
Compiling Pig to MapReduce
Unit 5 – Spark
Intro to Spark using PySpark
Basic Spark concepts: RDDs, transformations, actions
PairRDDs and aggregating transformations
Advanced Spark: partitions; shared variables
SparkSQL
Unit 6 – Project Week
Case studies/Final projects

Overview
This class is a comprehensive introduction to data analysis with the Python programming language. This class targets people who have some basic knowledge of programming and want to take it to the next level. It introduces how to work with different data structures in Python and covers the most popular data analytics and visualization modules, including numpy, scipy, pandas, matplotlib, and seaborn. We use Ipython notebook to demonstrate the results of codes and change codes interactively throughout the class.
Syllabus
Unit 1: Introduction to Python
Python is a high-level programming language. You will learn the basic syntax and data structures in Python. We demonstrate and run codes within Ipython notebook, which is a great tool providing a robust and productive environment for interactive and exploratory computing.
Introduction to Ipython notebook
Basic objects in Python
Variables and self-defining functions
Control flow
Data structures
Unit 2: Explore Deeper with Python
Python is an object-oriented programming (OOP) language. Having some basic knowledge of OOP will help you understand how Python codes work. More often than not, you will have to deal with data that is dirty and unstructured. You will learn many ways to clean your data such as applying regular expressions.
Introduction to object-oriented programming
How to deal with files
Run Python scripts
Handling and processing strings
Unit 3: Scientific Computation Tools
There are two modules for scientific computation that make Python powerful for data analysis: Numpy and Scipy. Numpy is the fundamental package for scientific computing in Python. SciPy is an expanding collection of packages addressing scientific computing.
Numpy
Scipy
Unit 4: Data Visualization
Python can also generate graphics easily using “Matplotlib” and “Seaborn”. Matplotlib is the most popular Python library for producing plots and other 2D data visualizations. Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing statistical graphics.
Seaborn
Matplotlib
Unit 5: Data manipulation with Pandas
Pandas provides rich data structures and functions for working with structured data. The “DataFrame” object in Pandas is just like the “data.frame” object in R. Pandas makes data manipulation (filter, select, group, aggregate, etc.) as easy as in R.
Pandas
Final Project
After 20 hours of structured lectures, students are encouraged to work on an exploratory data analysis project based on their own interests. A project presentation demo will be arranged afterwards.

NYC Data Science Academy offers 12 week data science bootcamps. In these programs, students learn beginner and intermediate levels of Data Science with R, Python, Hadoop & Spark, Github, and SQL as well as the most popular and useful R and Python packages like XgBoost, Caret, dplyr, ggplot2, Pandas, scikit-learn, and more.  Once the learning foundation has been set, students work on multiple projects through the bootcamp. Along the way, students are assisted in preparing for employment process through resume review and interview preparation. The program distinguishes itself by balancing intensive lectures with real world project work, and by the breadth of its curriculum. Throughout the program students work alone and in teams to create at least four projects that are showcased to employers through multiple channels; private on-campus hiring partner events, student blogs, meetups, and filmed presentations.
NYC Data Science Academy works closely with hiring partners and recruiting firms to create a pipeline of interest for its students. Ideal applicants should have a Masters or PhD degree in Science, Technology, Engineering or Math or equivalent experience in quantitative science or programming. Candidates with BA’s who have appropriate experience are also considered.

This course offers an accelerated intensive learning experience with Tableau – the growing standard in business intelligence for data visualization and dashboard creation. Without prior experience, students will learn to work with multiple data sources, create compelling visualizations, and roll out their data science products for continuous, scalable outputs to key stakeholders. By building insight and weaving narrative, students will be empowered to harness data in a striking way that provides value to organizations large and small.

Overview
This 20-hour course covers all the basic machine learning methods and Python modules (especially Scikit-Learn) for implementing them. The five sessions cover: simple and multiple Linear regressions; classification methods including logistic regression, discriminant analysis and naive bayes, support vector machines (SVMs) and tree based methods; cross-validation and feature selection; regularization; principal component analysis (PCA) and clustering algorithms. After successfully completing of this course, you will be able to explain the principles of machine learning algorithms and implement these methods to analyze complex datasets and make predictions.
Syllabus
Unit 1: Introduction and Regression
What is Machine Learning
Simple Linear Regression
Multiple Linear Regression
Numpy/Scikit-Learn Lab
Unit 2: Classification I
Logistic Regression
Discriminant Analysis
Naive Bayes
Supervised Learning Lab
Unit 3: Resampling and Model Selection
Cross-Validation
Bootstrap
Feature Selection
Model Selection and Regularization lab
Unit 4: Classification II
Support Vector Machines
Decision Trees
Bagging and Random Forests
Decision Tree and SVM Lab
Unit 5: Unsupervised Learning
Principal Component Analysis
Kmeans and Hierarchical Clustering
PCA and Clustering Lab
Final Project
After 20 hours of structured lectures, students are encouraged to work on an exploratory data analysis project based on their own interests. A project presentation demo will be arranged afterwards.

Overview
This course is a 35-hour program designed to provide a comprehensive introduction to R. You’ll learn how to load, save, and transform data as well as how to write functions, generate graphs, and fit basic statistical models with data. In addition to a theoretical framework in which you will learn the process of data analysis, this course focuses on the practical tools needed in data analysis and visualization. By the end of the course, you will have mastered the essential skills of processing, manipulating and analyzing data of various types, creating advanced visualizations, generating reports, and documenting your codes.
Prerequisites
Basic knowledge about computer components
Basic knowledge about programming
Syllabus
Unit 1: Basic Programming with R
Introduction to R
What is R?
Why R?
How to learn R
RStudio, packages, and the workspace
Basic R language elements
Data object types
Local data import/export
Introducing functions and control statements
In-depth study of data objects
Functions
Functional Programming
Unit 2: Basic Data Elements
Data transformation
Reshape
Split
Combine
Character manipulation
String manipulation
Dates and timestamps
Web data capture
API data sources
Connecting to an external database
Unit 3: Manipulating Data with “dplyr”
Subset, transform, and reorder datasets
Join datasets
Groupwise operations on datasets
Unit 4: Data Graphics and Data Visualization
Core ideas of data graphics and data visualization
R graphics engines
Base
Grid
Lattice
ggplot2
Big data graphics with ggplot2
Unit 5: Advanced Visualization
Customized graphics with ggplot2
Titles
Coordinate systems
Scales
Themes
Axis labels
Legends
Other plotting cases
Violin Plots
Pie charts
Mosaic plots
Hierarchical tree diagrams
scatter plots with multidimensional data
Time-series visualizations
Maps
R and interactive visualizations
Final Project
After 35 hours of structured lectures, students are encouraged to work on an exploratory data analysis project based on their own interests. A project presentation demo will be arranged afterwards.

Via analogy to biological neurons and human perception, this course is an introduction to artificial neural networks that brings high-level theory to life with interactive labs featuring TensorFlow, the most popular open-source Deep Learning library. Essential theory will be covered in a manner that provides students with an intuitive understanding of Deep Learning’s underlying foundations. Paired with hands-on code run-throughs in Jupyter notebooks as well as strategies for overcoming common pitfalls, this foundational knowledge will empower individuals with no previous understanding of neural networks to build production-ready Deep Learning applications across the major contemporary families: Convolutional Nets for machine vision; Long Short-Term Memory Recurrent Nets for natural language processing and time series analysis; Generative Adversarial Networks for producing realistic images; and Reinforcement Learning for playing video games.

Overview
This is a class for computer-literate people with no programming background who wish to learn basic Python programming. The course is aimed at those who want to learn “data wrangling” – manipulating downloaded files to make them amenable to analysis. We concentrate on language basics such as list and string manipulation, control structures, simple data analysis packages, and introduce modules for downloading data from the web.
Goals
This is a “short course” of four weeks, with five hours of class per week (split into 2 ½ hour evening classes). Classes will be given in a lab setting, with student exercises mixed with lectures. Students should bring a laptop to class. There will be a modest amount of homework after each class. Due to the focused nature of this course, there will be no individual class projects but the instructors will be available to help students who are applying Python to their own work outside of class.
Syllabus
Unit 1: List manipulation
Simple values and expressions
Defining functions, using ordinary syntax and lambda syntax
Lists
Built-in functions and subscripting
Nested lists
Functional operators: map and filter
List comprehensions
Multiple-list operations: map and zip
Functional operators: reduce
Unit 2: Strings and simple I/O
Characters
Strings as lists of characters
Built-in string operations
Input files as lists of strings
Print statement
Reading data from the web
Using the requests package
String-based web scraping (e.g. handling csv files)
Unit 3: Control structures
Statements vs. expressions
For loops
Variables in for loops
if statements
Simple and nested if statements
Conditional expressions in lambda functions
While loops
break and continue
Unit 4: Data Analysis Packages
NumPy
Ndarray
Subscripting and slicing
Operations
Pandas
Data Structure
Data Manipulation
Grouping and Aggregation

Overview
This 35-hour course introduces both the theoretical foundation of machine learning algorithms as well as their practical applications of machine learning techniques in R. It will introduce you to data mining, performance measures and dimension reduction, regression models, both linear and generalized, KNN and Naïve Bayes models, tree models, and SVMs as well as the Association Rule for analysis. After successfully completing of this course, you will be able to break down the mathematics behind major machine learning algorithms, explain the principles of machine learning algorithms, and implement these methods to solve real-world problems.
Syllabus
Unit 1: Foundations of Statistics and Simple Linear Regression
Understand your data
Statistical inference
Introduction to machine learning
Simple linear regression
Diagnostics and transformations
The coefficient of determination
Unit 2: Multiple Linear Regression and Generalized Linear Model
Multiple linear regression
Assumptions and diagnostics
Extending model flexibility
Generalized linear models
Logistic regression
Maximum likelihood estimation
Model interpretation
Assessing model fit
Unit 3: kNN and Naive Bayes, the Curse of Dimensionality
The K-Nearest Neighbors Algorithm
The choice of K and distance measure
Conditional probability: Bayes’ Theorem
The Naive Bayes’ Algorithm
The Laplace estimator
Dimension reduction
The PCA procedure
Ridge and Lasso regression
Cross-validation
Unit 4: Tree Models and SVMs
Decision trees
Bagging
Random forests
Boosting
Variable Importance
Hyperplanes and maximal margin classifier
Sort margin and support vector classifier
Kernels and support vector machines
Unit 5: Cluster Analysis and Neural Networks
Cluster analysis
K-means clustering
Hierarchical clustering
Neural networks and perceptrons
Sigmoid neurons
Network topology and hidden features
Back propagation learning with gradient descent
Final Project
After 35 hours of structured lectures, students are encouraged to work on an exploratory data analysis project based on their own interests. A project presentation demo will be arranged afterwards.