In proposing to add or subtract federal aid for student loans and other programs on the basis of how well colleges and universities do things like successfully enroll and graduate low-income students, deliver quality education, and help students find jobs and repay debt, President Obama has put his finger on something big: A major contributing factor to the skyrocketing cost of college is the virtual absence of the concept of value in the higher education marketplace.

The conventional wisdom about college is that you get what you pay for, that the higher the price tag, the better the education. But that’s not necessarily the case. There are plenty of colleges that are both good and less expensive.

But higher education consumers—students and their bill-paying parents—are mostly in the dark about such schools because there’s scant information available to them about what would seemingly matter most: whether colleges and universities educate their students successfully. There’s plenty of information about colleges’ status and resources available in commercial guides like U.S. News and World Report’s America’s Best Colleges, because the magazine’s ratings are based largely on things like spending per student, schools’ reputation among college administrators, and incoming students’ SAT scores.

There’s some information on results, such as graduation rates. But good luck trying to find out how much students actually learn over the course of their college careers. If such information were available to students and parents, many of them would select schools that offered the best education for the money—and thereby pressure other colleges and universities to provide more for the tuition they charge, which is what the Obama administration is demanding.

Public information about where students are taught the best and learn the most would force institutions to pay attention to results rather than to resources and reputation. It would bring the concept of value front and center in higher education and help make our vast national investment in higher education far more rationale than it is today. (Ironically, the U.S. News ranking system undermines the concept of value. Ten percent of its rating is based on spending per student, meaning that a college that produces strong results at less cost and passes part of its savings on to students in the form of lower tuitions would see its ranking in the magazine decline.)

Traditionally, there haven't been many reliable tools with which to measure teaching and learning in higher education; U.S. News focuses on resources and reputation because that’s the information it can collect. But that’s changing. One example is the National Survey of Student Engagement (NSSE), launched with funding from the Pew Charitable Trusts in 2000 and now administered by Indiana University. Colleges participating in NSSE give students an extensive Web-based survey about their college experiences, focusing on teaching practices and out-of-class qualities that research has found to correlate with learning—things like the number of books and lengthy papers assigned in their courses, how much of their coursework involves applying theories to practical problems, how much homework their teachers assign, and how often they discuss coursework outside of class with teachers and classmates. Since its founding, NSSE has gathered information from over three million students in the U.S. and Canada and spawned a similar survey of community college students. The organization says it has found little relationship between having a prominent brand name and teaching students well.

Another, more direct measure of the quality of undergraduate teaching is the Collegiate Learning Assessment, developed by the non-profit Council for Aid to Education and launched nationally in 2004-05. It gauges progress in students' critical thinking skills over a college career by having them analyze documents, critique arguments, and synthesize information in longish essays. Like NSSE, the CLA has used technology, including computer-scored essays, to make its assessments affordable. Colleges and universities can measure the teaching and learning in their classrooms with a substantial degree of accuracy for $6,300 per campus.

The problem is that CLA and NSSE can’t report their findings publicly. Colleges and universities administer the tests voluntarily and have control over the results. And most are loath to put them on public display because, as NSSE has revealed, reputation doesn’t necessarily align with results. If they were smart, the vast majority of the nation’s 4,300 colleges and universities would happily shift the “best colleges” debate to a discussion of where students learn the most, because they lack the big endowments, brand names, or selective admissions to compete successfully in the U.S. News wealth-fame-exclusivity sweepstakes. But so far, only a handful of schools have publicized their NSSE and CLA results.

If the Obama administration really wanted to shake up the cost equation in higher education, it would push for the public disclosure, school by school, of the sorts of information captured in measures like CLA and NSSE.

The national conversation on college costs and affordability misses a key point: Even if tuition rates were frozen at their current level, the majority of all students and families would still need to borrow and these borrowers, increasingly, are having a hard time repaying the debt. A 2011 Institute for Higher Education Policy report showed that two out of every five federal student loan borrowers were delinquent over the last five years. The policy question we need to ask, therefore, isn’t simply “can we contain college costs,” but also, with our national policy of financing education with consumer debt, “what we are doing to help students manage that debt successfully”?

The cold reality is that colleges can’t wholly eliminate debt for their students, but they can do a much better job of helping them manage said debt. Colleges could – and should- be more proactive in educating and empowering students to take control of the repayment experience. Many federal student loan borrowers are having repayment difficulty not because they don’t earn enough, but because they simply don’t know how to take advantage of the many options available to fit their monthly payment into their budget and keep out of delinquency and default. In fact, no federal student loan borrower who acts in good faith to repay the debt should ever default, given the numerous safety nets built into the federal program for almost any set of circumstances, from unemployment and underemployment, to economic hardship and disability, to low income but high debt – and the list goes on. Sad as it is, often times filling out a form (or not even knowing the form exists or where to get it) is the difference between a borrower ruining his credit and staying in good standing. Research by my nonprofit, American Student Assistance, showed that just proactively reaching out to alumni during the repayment term to explain payment options boosted successful repayment rates by as much as 50 percent.

Now, President Obama’s proposed “college scorecard” promises to put colleges’ student loan repayment rates in the bright spotlight like never before. If colleges want to really move the needle in that area, they need to take a hard look at how well they’re preparing their in-school students for, and supporting their alumni through, loan repayment. To be sure, this will require an attitude adjustment, since the vast majority of institutions can’t afford the costs of providing services to both the in-school and alumni population, leaving the student consumer to be served and advised, whether federal or private, by the lender or their representatives.

But if colleges are concerned about boosting their public image, providing better value to satisfy policymakers, and establishing goodwill with alumni, they can’t risk removing themselves from the critical task of providing education debt management services. Academic policy conversations about college costs and affordability are all well and good, but we also need to get down to brass tacks and focus on what’s immediately achievable and will have the most impact – let’s do a better job of proactively empowering and educating students to manage the debt.

The question on the table is how to control the costs of higher education. So let’s take a look at the factors that have been driving those costs skyward for a long time.

Let’s begin by stating flatly that what is most definitely not driving college costs up is instruction. Over the years, tuition as a proportion of college costs has been falling steeply (page 25). The proportion of money devoted to instruction has also been steadily falling for a long time (page 7). So let’s look at what has been rising.

1. The Country Club

Much of the cost-push comes from the students and parents. Increasingly, students choose colleges on the basis of student reports obtained through internet sites and the US News and World Report rankings of the amenities available—food, sports, living quarters, recreation and so on. Higher education institutions feel compelled to engage in a spiraling expenditure race to spend ever more on these amenities in order to get or keep market share.

2. The (Psychiatric) Clinic

With the advent of the Americans with Disabilities Act, a large proportion of students come to college expecting some form of psychiatric or related counseling services. Entire buildings are now required at medium-sized universities to house the people who provide such services. This elaborate web of counseling services is very expensive, and though only some benefit from it, everyone who pays tuition pays for it.

3. The Research Game

When I was in college, at a research university, some of the greatest research laboratories in the world were at AT&T (the Bell Laboratories), DuPont, IBM and other companies. The federal government funded a lot of research at universities, but the assumption behind federal funding was the government was there to simply assist a core function of the university. So, in its role as helper, it should not be paying the full cost of the research being funded. As a practical matter, that ended up meaning that the federal government might pick up all the direct costs of the research being done, but not its full share of the costs of the university’s infrastructure, management, support services or library systems. But now, the great private labs in commercial concerns are a shadow of their former selves, a victim of the need to show quarterly results to investors, the country is more dependent than ever on university research to drive the economy, and the universities, strapped for funding to back this increased demand, are forced to turn to revenues from tuition to subsidize their research program.

4. The Incentives for the Professoriate

At many of our great research universities as well as at institutions in the next tier, the incentives for the professoriate are to produce research, not good teaching. While the steady reduction in the number of instructional days per year is partly a result of the fact that the customers (the students and their parents) do not object to a reduction in instructional time (a point to which I will return below), it is also true that the professoriate does not object either. It gives them more time to do what they really are incented to do, which is more research.

5. Collapsing State Budgets

Not only was The Great Recession the most serious recession the country has had since the Great Depression, but it affected state and local budgets far more seriously than the country as a whole. Local budgets just plain collapsed because they were so dependent on revenues from real estate taxes, which also collapsed. State legislatures, required by their constitutions to produce balanced budgets, and facing burgeoning safety net expenditures and falling revenues, had to make very deep budget cuts. The cost of elementary and secondary education is a large fraction of all state budgets, and the collapse of local tax revenues for the schools meant that the states could not look to the localities to take up the slack on the school budgets. Almost all of the states decided to do the best they could to protect their elementary and secondary education budgets as much as possible by making savage cuts in their budgets for their state-supported higher education institutions. The argument for this choice was that basic education is clearly a state responsibility and would collapse if not funded by the state, but the benefits of higher education accrue mainly to the individual and therefore it is the individual who ought to take up the slack in tough times. The trade offered to the state universities and colleges in return for the slashed budgets was the right to raise tuition charges.

6. The Case of the Community Colleges

Community colleges are also in an awful crunch. Their state budgets were cut, often severely, for all the reasons just stated. But many students who were planning to go to state four-year institutions or to private colleges decided during the Great Recession, that they could not afford to do so, and applied instead to their community colleges. Those institutions found themselves with many more students and much lower budgets, and that’s where they are now, sometimes forced to turn qualified students away, something many have never had to do before.

7. The Case of the Crème de la Crème

And then there are the 20 or so great universities, and perhaps a like number of colleges, that are the rough equivalent of the original Rembrandts, Monets and Klimts in the current art market. These institutions are very much aware that the difference in prestige and lifetime income produced by graduation from the two levels of institutions is still far less than the difference in the cost of tuition, room and board. So, within, limits, they can charge whatever they wish … and they do.

The reader will note that not one of these upward pressures on the cost of college has any basis in the upward pressure on the cost of instruction. Not one. They are all contributing to a steady and steep decline in the productivity of American higher education.

The Obama administration is right to take this on. Whether they have got a good handle on the solution is another matter.

In my view, one cannot solve public policy problems unless one identifies and then addresses the incentives facing the actors. They do what they do because of the incentives they face. If you want to change the behavior, you have to change the incentives.

This is hardly the place for a detailed analysis of the incentives facing the multitude of actors in this case or for a fair consideration of the possible changes that might be made in those incentives to produce better outcomes for the American people. So, at the risk of appearing to be rather simplistic, I will hit some highlights.

Many students and their parents at middle tier institutions are selecting those institutions based on the amenities offered because they believe the employers who they hope will offer the graduates jobs will look at the credential they have picked up, and make few inquiries about the quality of education they have received. Recent studies show that undergraduates appear to be learning little or nothing in their first couple of years in college and probably not much more in their upper division courses. Why should they? Professors demand little of them, because they would rather be doing research, and they don’t demand much of their professors, because prospective employers don’t seem to care what they learn. If we want to fix this, major employers need to announce that they are going to instruct their recruiters to carefully consider students’ scores on standardized measures of student performance administered at college graduation, and government would then be well advised to announce that it is prepared to make funds available for the development of such measures. There are already stirrings in the higher education world to use the CLAS assessment this way and to develop other measures for this purpose and those activities could be accelerated. But none of that will matter very much unless employers change their behavior and I have seen no signs of that yet.

But these measures by themselves will not reduce the costs of the Country Club, which is a large and growing share of the tuition costs that everyone is complaining about. Here, too, I can’t see the institutions cutting back on these costs if they are competing for customers based on these amenities. But there is something that could be done that might make a real difference, not just with respect to the Country Club, but other components of the rising cost structure as well.

Suppose the federal government required all colleges and universities to publicly report their annual budgets in a uniform format, a format that breaks the costs into the following categories: instruction, research, amenities (the Country Club), health care and social/emotional counseling. There might be a few others, but it would be important have no more than ten or so, so the important is not obscured by the less important.

Let’s put the first and second suggestion together. Imagine now that employers were demanding standardized achievement data from graduates, good measures were available and a rising number of colleges were producing that data – making it available not just to students and employers, but also in the aggregate to the public. That would create strong incentives for the institutions to spend more of their revenues on instruction and it would also result in producing more of a balance between instruction and research with respect to the incentives operating on the faculties of the institutions.

Imagine also that the colleges and universities were producing budget data in the aggregate in the categories just described. That would enable parents and students to make decisions about which institutions they would apply to, based on the trade-offs the institutions were offering between amenities and instruction. If they wanted to spend more on the country club amenities than on instruction and were willing to indebt themselves for years to do so, then they could go for it with their eyes open. But, if they wanted to maximize the return on their investment in terms of lifetime income, or if they actually valued instruction over the Country Club package of amenities, they would know much more than they do now about how much the institution was putting into instruction and what that investment was producing in terms of the value added, as measured by student performance on standardized measures. If employers were actually signaling their intention to base hiring decisions at least in part on the latter data, this would all add up to a dramatic change in the incentives operating on the institutions, the faculty and the customers, and all of those changes would point in the direction of increasing investments in instruction and decreasing investments made in the Country Club package and social-emotional counselling.

The next big change I would recommend to the Obama administration would be the assumption by the federal government of the costs of the university research they sponsor. As I pointed out above, the current rules require that the universities share a large part of those costs. It is disingenuous for the government to complain about rising tuition costs while its own actions are contributing greatly to those increases. It is time for the federal government, not college students and their parents, to pay for the costs of government-sponsored research.

Next, it is time for the public and public policy makers to face up to the current state of higher education finance. Observers have pointed out that what is evolving is a tiered system of higher education that is increasingly segregated by the ability to pay. We have desperately underfunded open-admissions systems for poor and minority students. We have another system for the vanishing middle class and upper middle class that is divided between state-supported and private institutions. And then we have the research universities, again divided between the public and private lower tier and, finally the top 20 or so.

In my view, the government should not be setting caps on what students should pay for tuition, because I do not know how to do that in a way that is fair and rational. But I have no problems with the government putting caps on what it will pay for some of the amenities included in the Country Club package. If parents want to pay more for fancier dorms, climbing walls, giant stadiums, and blah, blah, blah, go for it, but don’t expect the taxpayer to pay for the resort factor in the higher education experience.

Next, as I said earlier, it is time for the federal government to pay its fair share of the research budget, which would result in a significantly smaller bill to parents and students for tuition.

And, finally, it is time for the federal government and the state governments to reconsider the effects of their actions on publicly-funded colleges and mid-and-low tier public universities. It is true that there are private benefits to a college education. But it is also true that the public benefits to having a highly educated workforce are large and swiftly rising. This country is turning a blind eye to that truth and we will pay dearly for that.

Some elements of this package are consistent with the Obama package. Some are not. I believe that all, when combined, would dramatically improve the efficiency, effectiveness and productivity of our higher education system. And that is precisely what we now need.

Let’s cut right to the chase — I have about the same chance of being picked up by the Boston Red Sox as a utility player as President Obama does of having his proposals to control college costs get through Congress this year. But looking at what the President proposed on Friday (in a raucous speech at the University of Michigan) through the lens of short-term Capitol Hill feasibility misses the significance of what Obama is up to. Just a few years ago, the ideas the President hinted at in last week’s State of the Union and is now describing in more depth were considered fringe topics, basically the province of a few wonks and reform-minded policymakers. Talk of improving productivity in higher education bordered on blasphemy. Now the President of the United States is on board.

Obama wants to provide more data to parents and students about what colleges cost and how their students do after graduation. He also wants to change how federal aid works in order to create incentives for schools to keep costs down and keep interest on federal student loans low. Most noteworthy is his attempt to catalyze innovations at colleges and universities to improve productivity and encourage states to reform higher education through a grant competition similar to his Race to the Top program that has led many states to adopt K-12 reforms in order to win federal dollars. More specifics on the higher-ed competition will accompany the President’s budget request in February.

It’s clear now, though, that the President is essentially flipping the playing field. Regulating colleges from Washington is complicated and often impractical — and that’s when it isn’t politically impossible. But by using the competitive approach, Obama hopes to sidestep the obstacles the higher-education lobby can throw up in Washington and instead give states cover to put forward bold ideas to get college costs under control and improve student outcomes. He’ll still have to get his ideas through Congress, but this approach is politically more feasible than a frontal assault on the higher-education establishment. And there are some key Republicans, including House Speaker John Boehner, who in the past have supported versions of what Obama is proposing.

The headaches here are implicit in the way that the President views higher ed. It's hardly fair to pin the blame on President Obama alone, however, because policymakers have spent decades wedging us into this corner. The normal way that free societies encourage "responsible" behavior when it comes to the cost of services is to allow sellers to set a price and buyers to decide whether they're willing to pay it. The resulting market dynamic creates pressure on self-interested sellers to moderate their pricing, and encourages buyers to pay attention to cost.

In higher education (like in health care), we've done our damndest to smother this dynamic. Massive state and federal subsidies ensure that tuitition at public two- and four-year institutions has little relationship to actual per pupil costs. Grants, loans, and subsidized interest rates mean that students have only a notional sense of what the true cost of higher ed is going to be. And massively discounted tuition for huge swaths of students means that prices don't mean much anyway.

One response is to try to start to foster more transparent data on price and quality. Happily, the administration is promoting some useful steps on this count. A second, more crucial step, would be to start to remove the subsidies and largesse that distort the higher ed market and make it only to easy for institutions to raise tuition without consequence. Unfortunately, here, President Obama simply promises more subsidies (discounted interest rates, calls for more state dollars) to complement the largesse he's previously championed (in ARRA, tucked into health care reform).

The frustrating result is, if you're not going to let market dynamics play out, you're stuck with efforts to insist on "responsible" behavior by fiat. And then we wind up with the President's finger-waving "you better behave" histrionics.