NYC finance benefited from slavery and the poor Irish were getting abused by the draft system of that time. Poor white areas in the south, in particular Appalachian areas, didn't support the south. Why is West Virginia a state?

All of that goes to show you it was about slavery. Like any big event in human history you can't say every single person chose sides based on one factor. However, the major players divided over slavery and that was the biggest outcome of the war.