I finished the course! I really enjoyed the learning experiences in Andrew's course so far. Let's see what I've learn for the two days! Overfitting - The Last Topic of this Course! Overfitting occurs when a machine learning model learns the details and noise in the training data to an extent that it negatively impacts the performance of the model on new data. This means the model is great at predicting or fitting the training data but performs poorly on unseen data, due to its inability to generalize from the training set to the broader population of data. The course explains that overfitting can be addressed by: We can't bypass underfitting. Overfitting and underfitting both are undesirable effects that suggest a model is not well-tuned to the task at hand, but they stem from opposite causes and have different solutions. Below two screenshots captured from course for my notes: Questions help me to master the content Words From Andrew At The End! I want to say congratulations on how far you've come and I want to say great job for getting through all the way to the end of this video. I hope you also work through the practice labs and quizzes. Having said that, there are still many more exciting things to learn. Awesome! I am already ready for next machine learning journeys!
Let's continue! Today is mainly learning about "Decision boundary", "Cost function of logistic regresion", "Logistic loss" and "Gradient Descent Implementation for logistic regression". We found out the "Decision boundary" is when z equals to 0 in the sigmod function. Because at this moment, its value will be just at neutral position. Andrew gave an example with two variables, x1 + x2 - 3 (w1 = w2 = 1) the decision bounday is the line of x1 + x2 = 3. I want to say "Cost function for logistic regression" is the most hard in week 3 so far I've seen. I haven't quite figured out why the square error cost function not applicable and where the loss function came from. I have to re-watch the videos again. The lab is also useful. This particular cost function in above is derived from statistics using a statistical principle called maximum likelihood estimation (MLE). Questions and Answers Some thoughts of today Honestly, it feels like it's getting tougher and tougher. I can still get through the equations and derivations alright, it’s just that as I age, I feel like my brain is just not keeping up. At the end of each video, Andrew always congratulates me with a big smile, saying I’ve mastered the content of the session. But deep down, I really think what he's actually thinking is, "Ha, got you stumped again!" However, to be fair, Andrew really does explain things superbly well. I hope someday I can truly master this knowledge and use it effortlessly. Fighting!
A break due to sick Oh boy... I was sick for almost two weeks 🤒 After several skirmish with the germs or virus or whatever that dared to challenge my immune system, I’m thrilled to announce not just my victorious return to health but also to the enthralling world of machine learning. It’s as if the universe paused my coding powers, storing them safely, until I could reclaim them with a vengeance. And reclaim I did! Today, I triumphantly completed week two's coding assessment, feeling like a coder reborn. Let's dive back into the data-drenched depths where we left off, shall we? 🚀 The first coding assessment I couldn't recall all of the stuff actually. It's for testing implementation of gradient dscent for one variable linear regression. I did a walk through previous lessons and I found this summary is really helpful: This exercise enhanced what I've learnt in this week. Getting into Classification I started the learning of the 3rd week. Looks like it will be more interesting. I made a few notes: Probability that y is 1;Given input arrow x, parameters arrow w, b. I couldn't focus too long on this. Need to pause after watching a few videos. Bye now.
Today I started with Choosing the learning rate, reviewed the Jupyter lab and learnt what is feature engineering. Choosing the learning rate The graph taugh in Choosing the learning rate is helpful when develping models: Feature Engineering Using your knowledge or intuition about the problem to design new features usually by transforming or combining the original features of the problem in order to make it easier for the learning algorithm to make accurate predictions. What is Feature Engineering. It's the end of this learning week?! Didn't expect that I reached the end of this learning week! I didn't spend much time on the labs. There are three labs in this learning week and I am afaid I might not be ready for the practice lab of this week. Will come back to udpate more details after my exercises.
Welcome back I didn't get much time working on the course in past 5 days!!! Finally resuming today! Today I reviewed Feature scaling part 1 and learned Feature scaling part 2 and Checking gradient descent for convergence. The difficulty of the course is getter harder, 20mins video, I spent double time and needed to checking external articles to get better understanding. Feature Scaling What are features and parameters in below formula? hat of Price = w1x1 + w2x2 + b. x1 and x2 are features, former one represents size of house, later one represents number of bedrooms. w1 and w2 are parameters. When a possible range of values of a feature is large, it's more likely that a good model will learn to choose a relatively small parameter value. Likewise, when the possible values of the feature are small, like the number of bedrooms, then a reasonable value for its parameters will be relatively large like 50. So how does this relate to grading descent? At the end of this video, Andrew explained that the features need to be re-scaled or transformed sl that the cost function J using the transfomed data would shape better and gradient descent can find a much more direct path to the global minimum. When you have different features that take on very different ranges of values, it can cause gradient descent to run slowly but re scaling the different features so they all take on comparable range of values. because speed, upgrade and dissent significantly. Andrew Ng One key aspect of feature engineering is scaling, normalization, and standardization, which involves transforming…
Today I spent 10 30 60 mins reviewing previous notes, just realized that's a lot. I am amazing 🤩 Today start with "Gradient descent for multiple linear regression". Gradient descent for multiple linear regression Holy... At the beginning Andrew throw out below and said hope you still remember, I don't: Why I couldn't recognize... Where does this come? I spent 30mins to review several previous videos, then found it... The important videos are: 1. Week 1 Implementing gradient descent, Andrew just wrote down below without explaining (he explained later) 2. Gradient descent for linear regression Holy! Found a mistake in Andrew's course!On above screenshot, Andrew lost x(i) at the end of the first line! WOW! I ROCK! Spent almost 60mins! I am done for today! The situation reversed an hour later But I felt upset, I was NOT convinced I found the simple mistake especially in Andrew's most popular Machine learning course! I started trying to resolve the fomula. And.... I found out I was indeed too young too naive... Andrew was right... I got help from my college classmate who has been dealing with calculus everyday for more than 20 years... This is the derivation process of the formula written by him: He said this to me like my math teacher in college: Chain rule, deriving step by step. If you still remember, I have mentined Parul Pandey’s Understanding the Mathematics behind Gradient Descent in my previous post Supervised Machine Learning – Day 2 & 3 – On My Way To Becoming A Machine Learning Person, in her post she…
So difficult to manage some time on this Day 4 was a long day for me, just got 15mins before bed to quickly skim through the video "multiple features" and "vectorization part 1". Day 5, a longer day than yesterday... went to urgent care in morning... then back-to-back meeting after come back.... lunch... back-to-back meeting again... need to step out again... Anyway, that's life. Multiple features (variables) and Vectorization In "multiple features", Andrew uses crispy language explained how to simplify the multiple features formula by using vector and dot product. In "Vectorization Part 1", Andrew introduced how to use NumPy to do dot product and said GPU is good at this type of calculation. Numpy function can use parallel hardware (like GPU) to make dot product fast. In "Vectorization Part 2", Andrew further introduced why computer can do dot product fast. He used gradient descent as an example. The lab was informative, I walked through all of them though I've known most of them before. Questions for helping myself learning I created the following questions to test my knowledge later. What is x(4)1 in above graph?
Supervised Machine Learning – Day 2 & 3 - On My Way To Becoming A Machine Learning Person
A brief introduction Day 2 I was busy and managed only 15 mins for "Supervised Machine Learning" video + 15 mins watched But what is a neural network? | Chapter 1, Deep learning from 3blue1brown. Day 3 I managed 30+ mins on "Supervised Machine Learning", and some time spent on articles reading, like Parul Pandey's Understanding the Mathematics behind Gradient Descent, it's really good. I like math 😂 So this notes are mixed. Notes Implementing gradient descent Notation I was struggling in writting Latex for the formula, then found this table is useful (source is here): Andrew said I don't need to worry about the derivative and calculas at all, I trust him, next I dived into my bookcases and found out my advanced mathmatics books used in college, and spent 15 mins to review, yes, I don't need. Snapped two epic shots of my "Advanced Mathematics" book used in my college time to show off my killer skills in derivative and calculus - pretty sure I've unlocked Math Wizard status! Reading online Okay. Reading some online articles. If we are able to compute the derivative of a function, we know in which direction to proceed to minimize it (for the cost function). From Parul Pandey, Understanding the Mathematics behind Gradient Descent Parul briefly introduced Power rule and Chain rule, fortunately, I still remember them learnt from colleage. I am so proud. After reviewing various explanations of gradient descent, I truly appreciate Andrew's straightforward and precise approach! He was kiddish some times drawing a stick man walking down a hill…
The Beginning As I've been advancing technologies of my AI-powered product knowlege base chatbot which based on Django/LangChain/OpenAI/Chroma/Gradio which is sitting on AI application/framework layer, I also have kept an eye on how to build a pipeline for assessing the accuracy of machine learning models which is a part of AI Devops/infra. But I realized that I have no idea how to meature a model's accuracy. This makes me upset. Then I started looking for answers. My first google search on this is "how to measure llm accuracy", it brought me to Evaluating Large Language Models (LLMs): A Standard Set of Metrics for Accurate Assessment, it's informative. It's not a lengthy article and I read through it. This opens a new world to me. There are standard set of metrics for evaluating LLMs, including: I don't know all of them and where to start! I have to tell meself, "Man, you don't know machine learning..." So my next search was "machine learning course", Andrew Ng's Supervised Machine Learning: Regression and Classification now came on top of the google search results! It's so famous and I knew this before! Then I made a decision, I want to take action now and finish it thoroughly! I immedially enrolled into the course. Now let's start the journey! Day 1 Started Basics 1. What is ML? Defined by Arthur Samuel back in the 1950 😯 "Field of study that gives computers the ability to learn without being explicitly programmed." The above claims gaves the key point (The highlighted part) which could answer the question from…