One thing that you've missed is health problems. A lot of poor people cannot work for psysical or psychological reasons.
That was said.
I think it is hard to argue that the poor have just as much opportunity as the rich. 1-2% of poor people can get scholarships - what number is that for middle class and rich? Significantly higher. And just because someone is not brilliant does not mean they should have no chance in college.
The thing is that someone at the 70th percentile who is wealthy is pretty much guaranteed a great job. The same person who is poor is guaranteed an extreme struggle.
There isn't much that can be done to change this beyond widen scholarship programs.