Do smaller people have to work harder than people who weigh more to lose weight?
In my experience, yes. I lost all but 10 lbs but that last ten is not coming off.
The graph (weight in relation to time) of someone losing weight usually looks like hyperbola. That means you are your weight loss rate is getting lower when you lose weight and at some point becomes practically zero.
So yes, people who weight more lose weight faster.