Should I leave Fordham?

Its been a confusing past two years for me, not only struggling to decide to go to Fordham, but now Im struggling to understand if this is where I want to be given my circumstances. I have always wanted to pursue IB, and I always knew I wanted to try to go to school in NYC. I foolishly decided to pursue Fordham with student loans (hefty ones) instead of community college for a year or two and guaranteed transfer to UVA (with tuition covered for both).

I am a sophomore student-athlete at Fordham, and I really wanted to pursue that opportunity and lifestyle, but I was quickly overwhelmed last year ( my freshman year ) with not only athletics but also with the huge commitment of having a girlfriend and obviously academic rigors. As a result of trying to take care of all three factors... I ate shit. I developed poor grades, I did poorly athletically, and the only thing that came out of that year alive was my relationship (to which we broke up three weeks ago) I am now at a point where Im really considering a change. The school's rankings are continuing to drop, the tuition is continuing to rise, I am not getting much $$$ from athletics, and financial aid is a joke. My parents did not support me coming here, and as a result, made me finance it on my own. They would, however, pay for me if I decided to go to UVA (family school).

I still have an insane desire to pursue IB; my grades this year so far have been stellar, I have networked my ass off and made a handful of good connections. My athletics are doing great, but when I sit down and think about the academic deficit I dug myself into, I wonder if its even worth/possible to not only drag that GPA back up, but continue to subject myself into incredible amounts of debt for a mediocre non/semi target school. 

I really do love Fordham as a school, I love being in the city, I love the environment and the people here/teamates. I'm trying to be open-minded, put that all behind me, and really make a decision for ME. What will be best for ME in the future? 

Please feel free to shoot some advice over regardless if its harsh or not. Transparency is best for me at this point.