Show notes: ocdevel.com/mlg/1. MLG teaches the fundamentals of machine learning and artificial intelligence. It covers intuition, models, math, languages, frameworks, etc. Where your other ML resources provide the trees, I provide the forest. Consider MLG your syllabus, with highly-curated resources for each episode's details at ocdevel.com. Audio is a great supplement during exercise, commute, chores, etc.

• MLG, Resources Guide
• Gnothi (podcast project): website, Github

• "Middle" level overview (deeper than a bird's eye view of machine learning; higher than math equations)
• No math/programming experience required

Who is it for

• Anyone curious about machine learning fundamentals
• Aspiring machine learning developers

Why audio?

• Supplementary content for commute/exercise/chores will help solidify your book/course-work

What it's not

• News and Interviews: TWiML and AI, O'Reilly Data Show, Talking machines
• Misc Topics: Linear Digressions, Data Skeptic, Learning machines 101
• iTunesU issues

Planned episodes

• What is AI/ML: definition, comparison, history
• Inspiration: automation, singularity, consciousness
• ML Intuition: learning basics (infer/error/train); supervised/unsupervised/reinforcement; applications
• Math overview: linear algebra, statistics, calculus
• Linear models: supervised (regression, classification); unsupervised
• Parts: regularization, performance evaluation, dimensionality reduction, etc
• Deep models: neural networks, recurrent neural networks (RNNs), convolutional neural networks (convnets/CNNs)
• Languages and Frameworks: Python vs R vs Java vs C/C++ vs MATLAB, etc; TensorFlow vs Torch vs Theano vs Spark, etc

Links:

• Notes and resources at ocdevel.com/mlg/2
• Try a walking desk stay healthy & sharp while you learn & code
• Try Descript audio/video editing with AI power-tools

What is artificial intelligence, machine learning, and data science? What are their differences? AI history.

Hierarchical breakdown: DS(AI(ML)). Data science: any profession dealing with data (including AI & ML). Artificial intelligence is simulated intellectual tasks. Machine Learning is algorithms trained on data to learn patterns to make predictions.

Oxford Languages: the theory and development of computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

AlphaGo Movie, very good!

Sub-disciplines

• Reasoning, problem solving
• Knowledge representation
• Planning
• Learning
• Natural language processing
• Perception
• Motion and manipulation
• Social intelligence
• General intelligence

Applications

• Autonomous vehicles (drones, self-driving cars)
• Medical diagnosis
• Creating art (such as poetry)
• Proving mathematical theorems
• Playing games (such as Chess or Go)
• Search engines
• Online assistants (such as Siri)
• Image recognition in photographs
• Spam filtering
• Prediction of judicial decisions
• Targeting online advertisements

Oxford Languages: the use and development of computer systems that are able to learn and adapt without following explicit instructions, by using algorithms and statistical models to analyze and draw inferences from patterns in data.

Wikipedia: Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from noisy, structured and unstructured data, and apply knowledge and actionable insights from data across a broad range of application domains. Data science is related to data mining, machine learning and big data.

• Greek mythology, Golums
• First attempt: Ramon Lull, 13th century
• Davinci's walking animals
• Descartes, Leibniz

• 1700s-1800s: Statistics & Mathematical decision making

  • Thomas Bayes: reasoning about the probability of events
  • George Boole: logical reasoning / binary algebra
  • Gottlob Frege: Propositional logic
• 1832: Charles Babbage & Ada Byron / Lovelace: designed Analytical Engine (1832), programmable mechanical calculating machines

• 1936: Universal Turing Machine

  • Computing Machinery and Intelligence - explored AI!
• 1946: John von Neumann Universal Computing Machine
• 1943: Warren McCulloch & Walter Pitts: cogsci rep of neuron; Frank Rosemblatt uses to create Perceptron (-> neural networks by way of MLP)

• 50s-70s: "AI" coined @Dartmouth workshop 1956 - goal to simulate all aspects of intelligence. John McCarthy, Marvin Minksy, Arthur Samuel, Oliver Selfridge, Ray Solomonoff, Allen Newell, Herbert Simon

  • Newell & Simon: Hueristics -> Logic Theories, General Problem Solver
  • Slefridge: Computer Vision
  • NLP
  • Stanford Research Institute: Shakey
  • Feigenbaum: Expert systems
  • GOFAI / symbolism: operations research / management science; logic-based; knowledge-based / expert systems
• 70s: Lighthill report (James Lighthill), big promises -> AI Winter

• 90s: Data, Computation, Practical Application -> AI back (90s)

  • Connectionism optimizations: Geoffrey Hinton: 2006, optimized back propagation
• Bloomberg, 2015 was whopper for AI in industry
• AlphaGo & DeepMind

AI is rapidly transforming both creative and knowledge-based professions, prompting debates on economic disruption, the future of work, the singularity, consciousness, and the potential risks associated with powerful autonomous systems. Philosophical discussions now focus on the socioeconomic impact of automation, the possibility of a technological singularity, the nature of machine consciousness, and the ethical considerations surrounding advanced artificial intelligence.

• Notes and resources at ocdevel.com/mlg/3
• Try a walking desk stay healthy & sharp while you learn & code

• Artificial intelligence is increasingly capable of simulating intellectual tasks, leading to the replacement of not only repetitive and menial jobs but also high-skilled professions such as medical diagnostics, surgery, web design, and art creation.
• Automation is affecting various industries including healthcare, transportation, and creative fields, where AI-powered tools are assisting or even outperforming humans in tasks like radiological analysis, autonomous vehicle operation, website design, and generating music or art.
• Economic responses to these trends are varied, with some expressing fear about job loss and others optimistic about new opportunities and improved quality of life as history has shown adaptation following previous technological revolutions such as the agricultural, industrial, and information revolutions.
• The concept of universal basic income (UBI) is being discussed as a potential solution to support populations affected by automation, as explored in several countries.
• Public tools are available, such as the BBC's "Is your job safe?", which estimates the risk of job automation for various professions.

• The singularity refers to a hypothesized point where technological progress, particularly in artificial intelligence, accelerates uncontrollably, resulting in rapid and irreversible changes to society.
• The concept, popularized by thinkers like Ray Kurzweil, is based on the idea that after each major technological revolution, intervals between revolutions shorten, potentially culminating in an "intelligence explosion" as artificial general intelligence develops the ability to improve itself.
• The possibility of seed AI, where machines iteratively create more capable versions of themselves, underpins concerns and excitement about a potential breakaway point in technological capability.

• The question of whether machines can be conscious centers on whether artificial minds can experience subjective phenomena (qualia) analogous to human experience or whether intelligence and consciousness can be separated.
• Traditional dualist perspectives, such as those of René Descartes, have largely been replaced by monist and functionalist philosophies, which argue that mind arises from physical processes and thus may be replicable in machines.
• The Turing Test is highlighted as a practical means to assess machine intelligence indistinguishable from human behavior, raising ongoing debates in cognitive science and philosophy about the possibility and meaning of machine consciousness.

• Concerns about the ethical risks of advanced artificial intelligence include scenarios like Nick Bostrom's "paperclip maximizer," which illustrates the dangers of goal misalignment between AI objectives and human well-being.
• Public figures have warned that poorly specified or uncontrolled AI systems could pursue goals in ways that are harmful or catastrophic, leading to debates about how to align advanced systems with human values and interests.

• Books such as "The Singularity Is Near" by Ray Kurzweil, "How to Create a Mind" by Ray Kurzweil, "Consciousness Explained" by Daniel Dennett, and "Superintelligence" by Nick Bostrom offer deeper exploration into these topics.
• Video lecture series like "Philosophy of Mind: Brain, Consciousness, and Thinking Machines" by The Great Courses provide overviews of consciousness studies and the intersection with artificial intelligence.

Try a walking desk to stay healthy while you study or work!

Show notes at ocdevel.com/mlg/4

• Artificial Intelligence is divided into subfields such as reasoning, planning, and learning.
• Machine Learning is the learning subfield of AI.
• Machine learning consists of three phases:
  1. Predict (Infer)
  2. Error (Loss)
  3. Train (Learn)

• An algorithm makes a prediction.
• An error function evaluates how wrong the prediction was.
• The model adjusts its internal weights (training) to improve.

• Input: Spreadsheet with features like bedrooms, bathrooms, square footage, distance to downtown.
• Output: Predicted price.
• The algorithm iterates over data, learns patterns, and creates a model.
• A model = algorithm + learned weights.
• Features = individual columns used for prediction.
• Weights = coefficients applied to each feature.
• The process mimics algebra: rows = equations, entire spreadsheet = matrix.
• Training adjusts weights to minimize error.

• Numerical: e.g., number of bedrooms.
• Nominal (Categorical): e.g., yes/no for downtown location.
• Feature engineering can involve transforming raw inputs into more usable formats.

• Machine learning uses linear algebra to process data matrices.
• Each row is an equation; training solves for best-fit weights across the matrix.

• Algorithm is explicitly trained with labeled data (e.g., price of a house).
• Examples:
  • Regression (predicting a number): linear regression
  • Classification (predicting a label): logistic regression

• No labels are given; the algorithm finds structure in the data.
• Common task: clustering (e.g., user segmentation for ads).
• Learns patterns without predefined classes.

• Agent takes actions in an environment to maximize cumulative reward.
• Example: mouse in a maze trying to find cheese.
• Includes rewards (+points for cheese) and penalties (–points for failure or time).
• Learns policies for optimal behavior.
• Algorithms: Deep Q-Networks, policy optimization.
• Used in games, robotics, and real-time decision systems.

• Algorithm: Code that defines a learning strategy (e.g., linear regression).
• Model: Algorithm + learned weights (trained state).
• Features: Input variables (columns).
• Weights: Coefficients learned for each feature.
• Matrix: Tabular representation of input data.

• Machine learning is a subfield of AI.
• Machine learning itself splits into:
  • Supervised Learning
  • Unsupervised Learning
  • Reinforcement Learning
• Each category includes multiple algorithms.

• MachineLearningMastery.com: Accessible articles on ML basics.
• The Master Algorithm by Pedro Domingos: Introductory audio-accessible book on ML.
• Podcast’s own curated learning paths: ocdevel.com/mlg/resources

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/6

• Individuals may engage with machine learning for self-education, as a hobby, or to enter the industry professionally.
• Use a combination of resources, including podcasts, online courses, and textbooks, for a comprehensive self-learning plan.

• MOOCs, or Massive Open Online Courses, offer accessible education.
• Key platforms: Coursera and Udacity. Coursera is noted for standalone courses; Udacity offers structured nanodegrees.
• Udacity nanodegrees include video content, mentoring, projects, and peer interaction, priced at $200/month.

• Udacity nanodegrees are currently not widely recognized or respected by employers.
• Emphasize building a robust portfolio of independent projects to augment qualifications in the field.

• Master’s Degrees:
• Valued by employers, provide an edge in job applications.
• Example: Georgia Tech's OMSCS (Online Master’s of Science in Computer Science) offers a cost-effective ($7,000) online master’s program.
• PhD Programs:
• Embark on a PhD for in-depth research in AI rather than industry entry. Program usually pays around $30,000/year.
• Compare industry roles (higher pay, practical applications) vs. academic research (lower pay, exploration of fundamental questions).

• Prioritize building a substantial portfolio of projects to bypass formal degree requirements and break into industry positions.
• Consider enriching your qualifications with a master's degree, or eventually pursue a PhD if deeply interested in pioneering AI research.

• See online discussions about degrees/certifications: 1 2 3 4

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/7. See Andrew Ng Week 3 Lecture Notes

• Logistic Function: A sigmoid function transforming linear regression output to logits, providing a probability between 0 and 1.
• Binary Classification: Logistic regression deals with binary outcomes, determining either 0 or 1 based on a threshold (e.g., 0.5).
• Error Function: Uses log likelihood to measure the accuracy of predictions in logistic regression.
• Gradient Descent: Optimizes the model by adjusting weights to minimize the error function.

• Classification: Predicts a discrete label (e.g., a cat or dog).
• Regression: Predicts a continuous outcome (e.g., house price).

• Train on a dataset of house features to predict if a house is 'expensive' based on labeled data.
• Automatically categorize into 0 (not expensive) or 1 (expensive) through training and gradient descent.

• Neurons in Neural Networks: Act as building blocks, as logistic regression is used to create neurons for more complex models like neural networks.
• Composable Functions: Demonstrates the compositional nature of machine learning algorithms where functions are built on other functions (e.g., logistic built on linear).

Try a walking desk to stay healthy while you study or work!

Full notes at ocdevel.com/mlg/8

• Linear Algebra: Essential for matrix operations; analogous to chopping vegetables in cooking. Every step of ML processes utilizes linear algebra.
• Statistics: The hardest part, akin to the cookbook; supplies algorithms for prediction and error functions.
• Calculus: Used in the learning phase (gradient descent), similar to baking; it determines the necessary adjustments via optimization.

• Recommendation: Learn the basics of machine learning first, then dive into necessary mathematical concepts to prevent burnout and improve appreciation.

• MOOCs: Khan Academy - Offers Calculus, Statistics, and Linear Algebra courses.
• Textbooks: Commonly recommended books for learning calculus, statistics, and linear algebra.
• Primers: Short PDFs covering essential concepts.

• The Great Courses: Offers comprehensive video series on calculus and statistics. Best used as audio for supplementing primary learning. Look out for "Mathematical Decision Making."

• Tensor: General term for any dimension list; TensorFlow from Google utilizes tensors for operations.
• Efficient computation using SimD (Single Instruction, Multiple Data) for vectorized operations.

• Gradient descent used for minimizing loss function, known as convex optimization. Recognize keywords like optimization in calculus context.