Artificial Neuron and Perceptron
#
knowledge in neural networks is stored in connection weights, and learning means modifying those weights.
Biological Neuron
#
A biological neuron is a specialised cell that processes and transmits information through electrical and chemical signals.
Core components:
- Dendrites: receive signals from other neurons
- Cell body (soma): processes incoming signals
- Axon: transmits the output signal
- Synapses: connection points between neurons
Biological intuition:
- many inputs arrive to one neuron
- one neuron can connect out to many neurons
- massive parallelism enables fast perception and recognition
Artificial Neuron
#
An artificial neuron is a simplified computational model inspired by biological neurons.
Machine learning Workflow
#
Data is the foundation of any machine learning system.
Quality of data matters more than model complexity.
Role of Data
#
Data determines:
- What patterns the model can learn
- How well it generalises
- Whether bias or noise is introduced
Bad data → bad model (even with perfect algorithms).
Data Preprocessing, wrangling
#
Raw data is never ready for training.
Data Issues
- Noise
- For objects, noise is an extraneous object
- For attributes, noise refers to modification of original values
- Use Log or Z Transfer to convert to mean
- Outliers
- Data objects with characteristics that are considerably different than most of the other data objects in the data set
- Handle: Use IQR method
- Find Lower and Upper Bound and replace Outlier with Lower or Upper Bound
- Missing Values
- Eliminate data objects or variables
- Handle: Estimate missing values
- Mean, Median or Mode
- Prefer Median if there are missing outliers
- Ignore the missing value during analysis
- Duplicate Data
- Major issue when merging data from heterogeneous sources
- Inconsistent Codes
- Find all Unique and transfer all inconsistent to
Data Preprocessing techniques
Linear Regression
#
Linear Regression is a supervised
ML
method used to predict a numerical target by fitting a model that is linear in its parameters.
In
ML
, linear models are a core baseline:
they’re fast, often surprisingly strong, and usually easy to interpret.
Key takeaway:
Linear Regression learns parameters by minimising a squared-error cost.
You can solve it directly (closed form) or iteratively (gradient descent),
and you can extend it using basis functions and regularisation.
February 21, 2026Direct solution method - Ordinary Least Squares and the Line of Best Fit
#
Revision:
OLS is the direct method for linear regression. It finds the best-fit line by minimising the sum of squared residuals without iterative updates.
Direct Method vs Iterative Method ☆
#
Linear regression parameters can be found in two main ways.
| Method | Main idea | When used |
|---|
| Ordinary Least Squares | Compute the best parameters directly | Small or moderate datasets |
| Gradient Descent | Start with parameters and update repeatedly | Large datasets or many features |
flowchart LR
A["Linear Regression"] --> B["Direct Solution<br/>OLS"]
A --> C["Iterative Solution<br/>Gradient Descent"]
B --> B1["Normal Equation"]
B --> B2["No learning rate"]
B --> B3["One-shot solution"]
C --> C1["Learning rate"]
C --> C2["Repeated updates"]
C --> C3["Stops after convergence"]
style A fill:#E1F5FE,stroke:#5b7db1,color:#000
style B fill:#C8E6C9,stroke:#5f8f6a,color:#000
style C fill:#FFF9C4,stroke:#b59b3b,color:#000
style B1 fill:#EDE7F6,stroke:#8a6fb3,color:#000
style B2 fill:#EDE7F6,stroke:#8a6fb3,color:#000
style B3 fill:#EDE7F6,stroke:#8a6fb3,color:#000
style C1 fill:#EDE7F6,stroke:#8a6fb3,color:#000
style C2 fill:#EDE7F6,stroke:#8a6fb3,color:#000
style C3 fill:#EDE7F6,stroke:#8a6fb3,color:#000
Why It Is Called “Least Squares” ☆
#
OLS is called least squares because it chooses parameters that make the squared residual errors as small as possible.
February 21, 2026 Cost Function
#
Revision:
A cost function converts model error into a single number. Training means changing the model parameters until this number becomes as small as possible.
Why Cost Function Matters in ML ☆
#
A machine learning model needs a way to decide whether one set of parameters is better than another.
For linear regression, every possible value of the parameters gives a different line.
The cost function tells us which line is better by measuring how far the predictions are from the true values.
February 21, 2026 Gradient Descent for Linear Regression
#
Revision:
Gradient descent is the step-by-step method for reducing the cost function when a direct closed-form solution is not convenient.
Where Gradient Descent Fits in ML ☆
#
Gradient descent is used when we want the model to learn parameters by repeatedly improving them.
For linear regression, it adjusts the slope and intercept until the prediction error becomes small.
flowchart LR
A["Initial Parameters"] --> B["Make Predictions"]
B --> C["Compute Cost"]
C --> D["Compute Gradient"]
D --> E["Update Parameters"]
E --> B
style A fill:#E1F5FE,stroke:#5b7db1,color:#000
style B fill:#C8E6C9,stroke:#5f8f6a,color:#000
style C fill:#FFF9C4,stroke:#b59b3b,color:#000
style D fill:#EDE7F6,stroke:#8a6fb3,color:#000
style E fill:#C8E6C9,stroke:#5f8f6a,color:#000
Core Idea ☆
#
The gradient tells us the direction in which the cost increases fastest.