# Python for Data Science For Dummies, 2ed

ISBN: 9788126524938

504 pages

## Description

Python is a general-purpose programming language that is popular with data scientists. It is free, as are a number of open-source libraries help acquire, organize, and process information. This book is designed for beginners to data analysis and covers the basics of Python data analysis programming and statistics. The book covers the Python fundamentals that are necessary to data analysis, including objects, functions, modules, and libraries. The book provides the statistical background needed to get started in data science programming, including probability, random distributions, hypothesis testing, confidence intervals, and building regression models for prediction.

Introduction

About This Book

Foolish Assumptions

Icons Used in This Book

Beyond the Book

Where to Go from Here

Part 1: Getting Started With Data Science and Python

Chapter 1: Discovering the Match between Data Science and Python

- Defining the Sexiest Job of the 21st Century
- Considering the emergence of data science
- Outlining the core competencies of a data scientist
- Linking data science, big data, and AI
- Understanding the role of programming
- Creating the Data Science Pipeline
- Preparing the data
- Performing exploratory data analysis
- Learning from data
- Visualizing
- Obtaining insights and data products
- Understanding Python's Role in Data Science
- Considering the shifting profile of data scientists
- Working with a multipurpose, simple, and efficient language
- Learning to Use Python Fast
- Loading data
- Training a model
- Viewing a result

Chapter 2: Introducing Python's Capabilities and Wonders

- Why Python?
- Grasping Python's Core Philosophy
- Contributing to data science
- Discovering present and future development goals
- Working with Python
- Getting a taste of the language
- Understanding the need for indentation
- Working at the command line or in the IDE
- Performing Rapid Prototyping and Experimentation
- Considering Speed of Execution
- Visualizing Power
- Using the Python Ecosystem for Data Science
- Accessing scientific tools using SciPy
- Performing fundamental scientific computing using NumPy
- Performing data analysis using pandas
- Implementing machine learning using Scikit-learn
- Going for deep learning with Keras and TensorFlow
- Plotting the data using matplotlib
- Creating graphs with NetworkX
- Parsing HTML documents using Beautiful Soup

Chapter 3: Setting Up Python for Data Science

- Considering the Off-the-Shelf Cross-Platform Scientific Distributions
- Getting Continuum Analytics Anaconda
- Getting Enthought Canopy Express
- Getting WinPython
- Installing Anaconda on Windows
- Installing Anaconda on Linux
- Installing Anaconda on Mac OS X
- Downloading the Datasets and Example Code
- Using Jupyter Notebook
- Defining the code repository
- Understanding the datasets used in this book

Chapter 4: Working with Google Colab

- Defining Google Colab
- Understanding what Google Colab does
- Considering the online coding difference
- Using local runtime support
- Getting a Google Account
- Creating the account
- Signing in
- Working with Notebooks
- Creating a new notebook
- Opening existing notebooks
- Saving notebooks
- Downloading notebooks
- Performing Common Tasks
- Creating code cells
- Creating text cells
- Creating special cells
- Editing cells
- Moving cells
- Using Hardware Acceleration
- Executing the Code
- Viewing Your Notebook
- Displaying the table of contents
- Getting notebook information
- Checking code execution
- Sharing Your Notebook
- Getting Help

Part 2: Getting Your Hands Dirty With Data

Chapter 5: Understanding the Tools

- Using the Jupyter Console
- Interacting with screen text
- Changing the window appearance
- Getting Python help
- Getting IPython help
- Using magic functions
- Discovering objects
- Using Jupyter Notebook
- Working with styles
- Restarting the kernel
- Restoring a checkpoint
- Performing Multimedia and Graphic Integration
- Embedding plots and other images
- Loading examples from online sites
- Obtaining online graphics and multimedia

Chapter 6: Working with Real Data

- Uploading, Streaming, and Sampling Data
- Uploading small amounts of data into memory
- Streaming large amounts of data into memory
- Generating variations on image data
- Sampling data in different ways
- Accessing Data in Structured Flat-File Form
- Reading from a text file
- Reading CSV delimited format
- Reading Excel and other Microsoft Office files
- Sending Data in Unstructured File Form
- Managing Data from Relational Databases
- Interacting with Data from NoSQL Databases
- Accessing Data from the Web

Chapter 7: Conditioning Your Data

- Juggling between NumPy and pandas
- Knowing when to use NumPy
- Knowing when to use pandas
- Validating Your Data
- Figuring out what's in your data
- Removing duplicates
- Creating a data map and data plan
- Manipulating Categorical Variables
- Creating categorical variables
- Renaming levels
- Combining levels
- Dealing with Dates in Your Data
- Formatting date and time values
- Using the right time transformation
- Dealing with Missing Data
- Finding the missing data
- Encoding missingness
- Imputing missing data
- Slicing and Dicing: Filtering and Selecting Data
- Slicing rows
- Slicing columns
- Dicing
- Concatenating and Transforming
- Adding new cases and variables
- Removing data
- Sorting and shuffling
- Aggregating Data at Any Level

Chapter 8: Shaping Data

- Working with HTML Pages
- Parsing XML and HTML
- Using XPath for data extraction
- Working with Raw Text
- Dealing with Unicode
- Stemming and removing stop words
- Introducing regular expressions
- Using the Bag of Words Model and Beyond
- Understanding the bag of words model
- Working with n-grams
- Implementing TF-IDF transformations
- Working with Graph Data
- Understanding the adjacency matrix
- Using NetworkX basics

Chapter 9: Putting What You Know in Action

- Contextualizing Problems and Data
- Evaluating a data science problem
- Researching solutions
- Formulating a hypothesis
- Preparing your data
- Considering the Art of Feature Creation
- Defining feature creation
- Combining variables
- Understanding binning and discretization
- Using indicator variables
- Transforming distributions
- Performing Operations on Arrays
- Using vectorization
- Performing simple arithmetic on vectors and matrices
- Performing matrix vector multiplication
- Performing matrix multiplication

Part 3: Visualizing Information

Chapter 10: Getting a Crash Course in MatPlotLib

- Starting with a Graph
- Defining the plot
- Drawing multiple lines and plots
- Saving your work to disk
- Setting the Axis, Ticks, Grids
- Getting the axes
- Formatting the axes
- Adding grids
- Defining the Line Appearance
- Working with line styles
- Using colors
- Adding markers
- Using Labels, Annotations, and Legends
- Adding labels
- Annotating the chart
- Creating a legend

Chapter 11: Visualizing the Data

- Choosing the Right Graph
- Showing parts of a whole with pie charts
- Creating comparisons with bar charts
- Showing distributions using histograms
- Depicting groups using boxplots
- Seeing data patterns using scatterplots
- Creating Advanced Scatterplots
- Depicting groups
- Showing correlations
- Plotting Time Series
- Representing time on axes
- Plotting trends over time
- Plotting Geographical Data
- Using an environment in Notebook
- Getting the Basemap toolkit
- Dealing with deprecated library issues
- Using Basemap to plot geographic data
- Visualizing Graphs
- Developing undirected graphs
- Developing directed graphs

Part 4: Wrangling Data

Chapter 12: Stretching Python's Capabilities

- Playing with Scikit-learn
- Understanding classes in Scikit-learn
- Defining applications for data science
- Performing the Hashing Trick
- Using hash functions
- Demonstrating the hashing trick
- Working with deterministic selection
- Considering Timing and Performance
- Benchmarking with timeit
- Working with the memory profiler
- Running in Parallel on Multiple Cores
- Performing multicore parallelism
- Demonstrating multiprocessing

Chapter 13: Exploring Data Analysis

- The EDA Approach
- Defining Descriptive Statistics for Numeric Data
- Measuring central tendency
- Measuring variance and range
- Working with percentiles
- Defining measures of normality
- Counting for Categorical Data
- Understanding frequencies
- Creating contingency tables
- Creating Applied Visualization for EDA
- Inspecting boxplots
- Performing t-tests after boxplots
- Observing parallel coordinates
- Graphing distributions
- Plotting scatterplots
- Understanding Correlation
- Using covariance and correlation
- Using nonparametric correlation
- Considering the chi-square test for tables
- Modifying Data Distributions
- Using different statistical distributions
- Creating a Z-score standardization
- Transforming other notable distributions

Chapter 14: Reducing Dimensionality

- Understanding SVD
- Looking for dimensionality reduction
- Using SVD to measure the invisible
- Performing Factor Analysis and PCA
- Considering the psychometric model
- Looking for hidden factors
- Using components, not factors
- Achieving dimensionality reduction
- Squeezing information with t-SNE
- Understanding Some Applications
- Recognizing faces with PCA
- Extracting topics with NMF
- Recommending movies

Chapter 15: Clustering

- Clustering with K-means
- Understanding centroid-based algorithms
- Creating an example with image data
- Looking for optimal solutions
- Clustering big data
- Performing Hierarchical Clustering
- Using a hierarchical cluster solution
- Using a two-phase clustering solution
- Discovering New Groups with DBScan

Chapter 16: Detecting Outliers in Data

- Considering Outlier Detection
- Finding more things that can go wrong
- Understanding anomalies and novel data
- Examining a Simple Univariate Method
- Leveraging on the Gaussian distribution
- Making assumptions and checking out
- Developing a Multivariate Approach
- Using principal component analysis
- Using cluster analysis for spotting outliers
- Automating detection with Isolation Forests

Part 5: Learning From Data

Chapter 17: Exploring Four Simple and Effective Algorithms

- Guessing the Number: Linear Regression
- Defining the family of linear models
- Using more variables
- Understanding limitations and problems
- Moving to Logistic Regression
- Applying logistic regression
- Considering when classes are more
- Making Things as Simple as Naïve Bayes
- Finding out that Naïve Bayes isn't so naïve
- Predicting text classifications
- Learning Lazily with Nearest Neighbors
- Predicting after observing neighbors
- Choosing your k parameter wisely

Chapter 18: Performing Cross-Validation, Selection, and Optimization

- Pondering the Problem of Fitting a Model
- Understanding bias and variance
- Defining a strategy for picking models
- Dividing between training and test sets
- Cross-Validating
- Using cross-validation on k folds
- Sampling stratifications for complex data
- Selecting Variables Like a Pro
- Selecting by univariate measures
- Using a greedy search
- Pumping Up Your Hyperparameters
- Implementing a grid search
- Trying a randomized search

Chapter 19: Increasing Complexity with Linear and Nonlinear Tricks

- Using Nonlinear Transformations
- Doing variable transformations
- Creating interactions between variables
- Regularizing Linear Models
- Relying on Ridge regression (L2)
- Using the Lasso (L1)
- Leveraging regularization
- Combining L1 & L2: Elasticnet
- Fighting with Big Data Chunk by Chunk
- Determining when there is too much data
- Implementing Stochastic Gradient Descent
- Understanding Support Vector Machines
- Relying on a computational method
- Fixing many new parameters
- Classifying with SVC
- Going nonlinear is easy
- Performing regression with SVR
- Creating a stochastic solution with SVM
- Playing with Neural Networks
- Understanding neural networks
- Classifying and regressing with neurons

Chapter 20: Understanding the Power of the Many

- Starting with a Plain Decision Tree
- Understanding a decision tree
- Creating trees for different purposes
- Making Machine Learning Accessible
- Working with a Random Forest classifier
- Working with a Random Forest regressor
- Optimizing a Random Forest
- Boosting Predictions
- Knowing that many weak predictors win
- Setting a gradient boosting classifier
- Running a gradient boosting regressor
- Using GBM hyperparameters

Part 6: The Part of Tens

Chapter 21: Ten Essential Data Resources

- Discovering the News with Subreddit
- Getting a Good Start with KDnuggets
- Locating Free Learning Resources with Quora
- Gaining Insights with Oracle's Data Science Blog
- Accessing the Huge List of Resources on Data Science Central
- Learning New Tricks from the Aspirational Data Scientist
- Obtaining the Most Authoritative Sources at Udacity
- Receiving Help with Advanced Topics at Conductrics
- Obtaining the Facts of Open Source Data Science from Masters
- Zeroing In on Developer Resources with Jonathan Bower

Chapter 22: Ten Data Challenges You Should Take

- Meeting the Data Science London + Scikit-learn Challenge
- Predicting Survival on the Titanic
- Finding a Kaggle Competition that Suits Your Needs
- Honing Your Overfit Strategies
- Trudging Through the MovieLens Dataset
- Getting Rid of Spam E-mails
- Working with Handwritten Information
- Working with Pictures
- Analyzing Amazon.com Reviews
- Interacting with a Huge Graph

Index