I think it makes sense to judge people based on where they go to college. For the most part, people who attend top universities (ivy league-- esp HYP, MIT, Oxford, Cambridge, Stanford, Caltech, Duke, WashU, Northwestern, UChicago, UC Berkeley, UCLA, etc) are far superior to students attending colleges of mediocre rank (such as most big state schools).
Students at these top schools have it all: academic excellence (high grades, high test scores), extracurricular activities (sports, music, art, etc), leadership involvement, and other compelling and unique sides to them to make them attractive candidates in a grueling and extremely competitive admissions process.
I currently attend UC Berkeley as an engineer, and I'm blown away at the quality of my peers. Many of my peers have razor sharp brains and are extremely hard workers. In addition to that, I know many of my friends who are also involved with campus sports, music groups (I myself play the electric guitar in an alternative rock band), leadership activities (I have friends who lead political clubs on campus), and more. We work hard for sure, but we also have social lives and we play hard: like everyone else, we go to parties, date, socialize, etc (UC Berkeley has a decent Greek life). We will also get excellent jobs once we graduate because top companies actually come to campus to recruit our graduates (same with Stanford and other top ranked schools).
In my view, students at top schools tend to have a strong work ethic, tend to be competitive while being cooperative (a trait that will help them in life), are ambitious and want to be leaders in the world, and genuinely care about learning about the world and have highly developed passions. Also many aren't "nerdy" as the media portrays us out to be: we have romantic relationships, we are social and outgoing and chill, we play sports (and some top schools have top-notch sports teams in addition to academic excellence, like Duke, Stanford, UC Berkeley swimming, etc). Therefore, students at top schools tend to be very well rounded, and are likely to be superior to students at lower ranked schools.
Students at top schools tend to be disciplined, have good time management, are focused, have a desire to be productive and hard working, and have strong passions that they want to pursue. Moreover, they want to change the world and be leaders for the better. I know many students who as little children were steps above their classmates and developed good study habits, good social skills, musical and athletic talents, and more. And you can be both very skilled as a student and have a social life (skilled student =/= socially awkward nerd). So saying that students at lower ranked schools may have better social skills isn't necessarily true (there are many "losers" at lower ranked schools: people who suck at academics and also at life in general, like having a relationship, being social, etc). Moreover, even if someone is very outgoing at a lower-ranked school, they might not develop oral and communication skills necessary to be charismatic in a professional setting, something that top-ranked business programs teach people to do (I know socially awkward people who majored in business at Berkeley and became some of the most outgoing and smooth people once they graduated). So companies would prefer someone who is professionally smart and charismatic over someone who is street and socially smart, but not professionally smart.
I have a friend who transferred to UC Berkeley from UC Riverside. He told me that Cal is superior in pretty much every aspect: better location, more interesting culture, more interesting students, better sports, better academics, world renowned professors, and more. He told me that the students at Cal are a shitton more academically motivated and take studying/learning very seriously and are disciplined. He also said that Cal students are way more involved in activities and leadership, such as community service, political groups, and extracurricular groups (taiko drumming, intramural sports, etc). You can't say students are top schools are necessarily "nerdy, shy, and socially awkward" when so many ppl at Berkeley are active in extracurricular like political activism. He also said that the students at Berkeley were more open minded and friendly than those at UC Riverside (where students according to him were a bit more socially conservative and cold, and many students didn't give a crap about school and goofed off, didn't show up at lecture, and dropped out).
True, students at lower ranked schools may have non-academic passions they like, like Photoshop, art, sports, music, and more. But students at top colleges are good that these IN ADDITION to being excellent in academics and being ambitious. So they are superior by being more well-rounded and having more skills.
Therefore when someone says he goes to Columbia or Harvard, that person deserves more respect and is a superior student to someone who attends University of Florida. Moreover, the Harvard grad is likely to be much more economically valuable than a Florida grad since the former will likely have much better skills, skills that employers will find attractive. Also being a Harvard grad is a turn on to the ladies, so that guy won't have trouble in his dating life.
This excludes people who had the ability to attend high ranked schools but opted to go to a lower ranked school b/c of financial and/or personal reasons. It also excludes legacy admits to top ranked schools.
Also one could say that there are students who are late bloomers at lower ranked colleges, people who have a lot of potential in life but didn't realize it until college (maybe they slacked off in high school but had the ability to be successful if the wanted to). But then again, isn't someone who was consistently a hard worker (someone who has been an excellent student all of his/her life and worked hard through high school) superior to someone who recently had their eyes opened about the value of education, and only recently tried to change his/her ways for the better?