0% COMPLETE

Course Curriculum

Product Sense via Machine Learning

Intro to Insights

Regressions(Preview)

Decision Trees

Partial Dependence Plots - Theory

Partial Dependence Plots - Practice

Product and Metrics - Case Studies

How to improve a given metric

Marketplace - Focusing on supply or demand?

Actionable insights vs spurious relationships

Behavioral vs demographic variables

How to create key growth metrics

How to come up with new feature ideas?

How to estimate user LifeTime Value?

User Similarity

Design metrics for customer service

Product Development - Should we test this new feature?

Metrics - Understanding what drives them

Metrics - How to find bugs

How to predict user behavior?

Metrics - How to create an effective metric

Metrics - Self-selection bias

Metrics - Trade-off between short term vs long term effects

How to improve revenue from ads

Metrics - How to create an effective metric for gaming apps

Personalization

Personalization - Full project

Personalization from a probabilistic standpoint

Unbalanced Classes

Cut-off point optimization

Changing class weights

Exercise

Missing data

Missing Data - Full Project

Missing Data - Uber example

Fraud and Machine Learning - Case Studies

Fake Listings

Random Forest - Key Params

False Positives vs False Negatives

Training/Test set split

Fraud - Two step authentication

Fraud - Fake Profiles

Supervised vs Unsupervised ML

A/B Testing - Practice

Overview

Sample Size Estimation

Randomization(Preview)

Randomization - Exercise

Novelty Effect(Preview)

General Guidelines

A/B testing - Case Studies

Test by Market

Opportunity costs

Testing long term metrics

Novelty Effect

Test by market - Drawbacks

Multiple A/B tests at the same time

Network Effect

Sample Size

Test first version of a data product

Statistically sound A/B test

When to rerun the same test again

When not to run an A/B test

New Product Case Studies - Miscellaneous

INSIGHTS: Should users be asked for their CC info when starting a free trial?(Preview)

INSIGHTS: How much would you charge for a new product?

INSIGHTS: What was the data-driven hypothesis behind testing FB stories?

INSIGHTS: Trade-off between revenue and user experience metrics

INSIGHTS: What did you do when you got unexpected results?

INSIGHTS: How would you place ads on a page?

INSIGHTS: Would you test a feature allowing users to easily switch account?(Preview)

INSIGHTS: Would you merge two similar products or run them separately?

INSIGHTS: Categorical variables in regressions. Better one-hot-encoding or multiple regressions?

INSIGHTS: How would you make Airbnb booking process smoother?

INSIGHTS: Can a bug actually improve your metric? What to do if that happens?

INSIGHTS: How to improve a marketplace?

A/B TESTING: Can you do a t-test on a dummy variable like conversion (0/1/)?

A/B TESTING: How would you test the success of a new ad campaign?

A/B TESTING: Define test statistical significance in layman’s terms. Why 0.05?

A/B TESTING: What would you do if test p-value is slightly above signficance level?

A/B TESTING: What to do if after a test some metrics are up and some down?

A/B TESTING: How to test different prices?

A/B TESTING: When would you design an A/B test as a sequence of multiple tests?

A/B TESTING: How can you halve the width of a t-test confidence interval?

A/B TESTING: How to check if a metric is actually up after a successful test?

A/B TESTING: How to avoid the A/B test issue of over-optimizing for the current user base?

A/B TESTING: When would you increase a test significance level?

METRICS: Conversion rate down, but absolute number of conversions up. Is it good or bad?

METRICS: How choosing an average-based metric vs a percentile one would impact product development?

METRICS: When to run a test on multiple metrics?

METRICS: When is accuracy a bad metric for a model?

METRICS: What kind of analysis is required after a logging-related bug?

METRICS: What kind of analysis is required after a product-related bug?

METRICS: In most metrics, you expect the average to be larger or smaller than the median?

Live Assistant App

App example with video recording

Projects with Solutions

Project: How to improve conversion rate

Project: Predicting fraud

Project: Predicting employee retention

Collection of tech company blog posts/case studies

A/B testing overview from Netflix, Scribd, MS

Theory behind A/B test from Google, StitchFix, Stackoverflow

A/B testing with connected users from Google, Instacart

Common Issues in A/B testing from Wallmart Labs, Google

Product Development from Airbnb, FB, Linkedin, Google, Netflix

Data Challenges with Solutions

Introduction

INSIGHTS: Workplace Diversity Analysis

Solution: Workplace Diversity Analysis

INSIGHTS: Funnel Analysis

Solution: Funnel Analysis

INSIGHTS: Subscription Retention Rate

Solution: Subscription Retention Rate

INSIGHTS: Sessionize user activity

Solution: Sessionize user activity

INSIGHTS: Video Sharing Analysis

Solution: Video Sharing Analysis

METRICS: Ads Analysis

Solution: Ads Analysis(Preview)

METRICS: Hotel Search Data

Solution: Hotel Search Data

A/B Testing: User Referral Program

Solution: User Referral Program

A/B TESTING: Pricing Test

Solution: Pricing Test

ML: Applying for a loan

Solution: Applying for a loan

ML: Optimization of Employee Shuttle Stops

Solution: Optimization of Employee Shuttle Stops

ML: Clustering Grocery Items

Solution: Clustering Grocery Items

ML: Credit Card Transactions

Solution: Credit Card Transactions

ML: Song Recommendation

Solution: Song Recommendation

Metrics - SQL Exercises

New Coding Course

Metrics: Time delta between consecutive events

SOLUTION: Time delta between consecutive events

Metrics: Segment by mobile, web, and cross-device users

SOLUTION: Segment by mobile, web, and cross-device users

Metrics: Identify power users based on their history

SOLUTION: Identify power users based on their history

Metrics: Estimate total and running values

SOLUTION: Estimate total and running values

Metrics: Summary stats - Custom implementation of the median vs average

SOLUTION: Summary stats - Custom implementation of the median vs average

Metrics : Rank users within groups

SOLUTION: Rank users within groups

A/B Testing - Practice

Randomization

Printing and PDF export are disabled for this lesson.