Differentiation With Respect To A Random Variable

I posted this a while ago, but I thought I would put it here as well. It is for a lot of people an obscure mathematical distinction that taking the derivative with respect to a random variable is not well-defined in general.