The difference is that the Jews were generally alright financially.
Some small long-established communities were (at least
on average), but they were the exception. Even if we ignore the situation in the east (where things were always pretty horrifying in material terms), Jewish communities in the more developed parts of Europe before the War tended to be concentrated in well defined inner city districts (Leopoldstadt or - parts of - the East End of London) and mostly worked in a handful of industries (IIRC for quite a time a majority of British Jews in employment worked in the clothing industry) and generally for Jewish employers, this not entirely by choice. My grandfather (who was not Jewish but Welsh-speaking Welsh) had a light sampling of some of these problems when he lived in the East End in the 1930s simply because his surname looked like it might be Jewish. That's just how things were...