A Markov Decision Process (MDP) is an extension of a Markov chain that incorporates decision-making to model sequential decision problems under uncertainty. While a standard Markov chain describes a system that transitions between states based on fixed probabilities, an MDP introduces actions, which allow an agent to influence state transitions. Each action taken in a given state leads to a new state according to a probabilistic transition model and is associated with a reward or cost. The goal in an MDP is to determine a policy—a mapping of states to actions—that maximizes the cumulative reward over time. Unlike a Markov chain, which is purely descriptive, an MDP is prescriptive, focusing on optimizing outcomes through strategic decisions.