Future Thinking with AI

July 29, 2018

AI will enhance search to create interactive reasoning and analytical systems. Search engines today do not know “why” we want some information and hence cannot reason about it. They also do not interact with us to help with analysis. An AI system that collects information based on knowing why it is needed and then asks more questions to refine its search would be clearly available well before 2030. These “search-thinking-bots” will also write up analyses based on parameters elicited from the conversation, and imbue these analyses with different political (left/right) and linguistic (aggressive/mild) slants, chosen by the human, using advances in language generation, which are already well underway. These “intellectual” agents will become companions, helping us make sense of our information overload. I often collect files of material on my cloud drive that I found interesting or needed to read later, and these agents would be able to summarize and engage me in a discussion of these materials, very much like an intellectual companion. It is unclear to me if I would need just one such agent, though it seems likely that different agents with diverse personalities may be more interesting! As always, we should worry what the availability of such agents might mean for normal human social interaction, but I can also see many advantages in freeing up time for socializing with other humans as well as enriched interactions, based on knowledge and science, assisted by our new intellectual companions.

Database technology as we know it will cease to exist. Modern protocols will automatically keep and tag data irrespective of which cloud it resides on. We will be able to obtain a view of all our data as all storage and devices will be linked. SQL and NoSQL databases will long be extinct. Instead, some modern version of knowledge graphs will be implemented. These new “information structures” will replace geo-spatial-temporal relationships with relationships based on concepts and context. Legal frameworks will adapt to allow our data to be fully owned by us. Everything will be homomorphically encrypted with new mathematical algorithms, allowing us to reveal different views of our data to specific entities, in the same way as we compartmentalize non-digital human interaction today.

Technological revolutions improve the world not because they offer cool new toys but because they improve lives with the better use of information. The distribution of means (broadly defined as wealth) is strongly affected by the distribution of knowledge and the use of information and AR/VR systems will become widespread, leveling the knowledge playing field. While the distribution of means has become wider (i.e., the proliferation of inequality of means/wealth), it is clear that the entire distribution has also shifted to the right. I think of this as Phase 1. Eventually, Phase 2 will see greater equality in the distribution of knowledge, followed by greater equality in the distribution of means. The risk to this view lies in political systems and thought not evolving quickly enough. These systems implement control through inequalities in knowledge, which lead to inequalities in wealth. Advances in technology unaccompanied by enlightened politics may delay progress and create turmoil in the short run. It may take a mutiny by a tech elite to move things forward in the right direction.



February 18, 2018

An academic life is complicated. It comes with a lot of flexibility and freedom, but also a lot of responsibility. With great flexibility comes huge complexity and variety. Let me explain.

A professor has may roles. First, a researcher, working usually on several projects at the same time with many co-authors, usually spread out all over the globe nowadays. Second, teaching, which itself involves many activities, from preparing classes to teaching in class, to setting homework and exams, to meeting students, advising, dealing with co-teachers, scheduling, etc. Third, university service, which includes many activities such as department chair, program head, committees of all type, raising money, and several other time sinks—like P.R., which professors are terrible at—and many other administrative make-work tasks, that have no productive purpose, but which universities excel at perpetuating. Fourth, editorial and referee work, which in itself seems light but is deceptively time-consuming. Fifth, travel for conferences, requiring prep before and catch up after. Sixth, consulting work, to make ends meet, and then also, pro bono work. It’s multifaceted and crazily so, especially if you are a fully functional professor.

So, okay you say, even people in the corporate world have to do many things. The problem in academia is this—these activities are being performed all the time and one ends up trying to do all of them at the same time. It’s multitasking at an extreme—I call it “supertasking.”

This is not to say that school teachers are not faced with the same supertasking chaos—they are. It’s fewer activities maybe, but much more people handling because school kids are as chaotic as they come. Fortunately, we professors have less of that, and I am grateful for that.

There is this notion that all professors do is teach. Nothing else. When people find out that the average professor teaches about a 100 hours a year, they wonder what we do with all the free time! They just don’t know. A junior professor might spend more than half her time on research, but as you age in the profession, that becomes harder and harder and dwindles to a much smaller number. Sometimes to zero when you become a dean for example, and you become Chief Cat Herder, Money Raiser, and Complaint Box. For those of us who love our research, that becomes a source of burnout. It’s when the profession ages you if you do not learn to handle it well.

But many professors are able to find the right balance and keep up their research despite the arduous pressure of supertasking. They keep going, doing their research while aggressively guarding their time. And being unavailable and upsetting those who think they should be always available.

People assume that professors know a lot more than we actually do and that our calling in life is to be on call. Because presumably, everyone knows that all we do is teach. I get emails saying stuff like—“I found your name on an internet search and your research is exactly what places you in a position to solve my job problem. I will call you tomorrow to discuss this.” No. Stop. We supertaskers are already working on too much and have no time, like new parents, especially mothers. And we have our own students, who get priority. And in any case, assuming we know a lot about everything is wrong. Free advice is not cheap in the long run. Stop. I know it feels good to dole out advice, and we do get an ego boost, but it’s irresponsible!

Managing supertasking is the key to quality work. When one does, it’s a fantastic creative experience. It makes up for the lousy pay in our profession. In the end, all professors want is a downgrade from supertasker to multitasker. Hard to hope for much more.  We need help, leave us alone for a while. Promise we won’t get up to mischief!


Oregon Road Trip

July 15, 2015
(by SR)
We set out from San Jose on a Sunday morning, so there was little traffic to contend with. Our ultimate destination was Crater Lake up in Oregon, and we’d already made hotel bookings at various stops along the way, because neither of us had seen that part of the state before. Also, we found that by pre-planning this aspect, we could avoid the impending July 4th surge!!
Our first stop was Mount Shasta. [Pics: 1 2] It was a 300 mile journey at the end of which we stayed at the Best Western Tree House. If you do go to Shasta, we recommend you make a stop for dinner at Lily’s, an organic restaurant down town, which seems pleasantly progressive when you are surrounded in such rustic simplicity!! Also visit Lake Siskiyou (great circular hike around for 7 miles) and pass through the town of Weed, CA!
The following day we left California and headed to the city of Klamath Falls, which is about another 85 miles north and is in Oregon. We stayed at the Holiday Inn Resort up there, which was actually really quiet, set on a golf course, and close to Klamath Lake. [Pics: 1 2] (In fact, it was not like a Holiday Inn at all) The best part about it was the bird watching around a marsh area actually on the resort premises. We stayed there for two days, and enjoyed the nature walks and the woodpeckers outside the room. Just make sure you take mosquito repellant!
While staying at Klamath Falls, we drove up to Crater Lake [Pics: 1 2 3 4], which is about an hour’s drive. As you approach the crater, you have a couple of choices as to how to approach the drive. (You cannot walk around it, you have to drive.) We drove clockwise around the lake (33 miles, the west rim drive), taking advantage of the multiple rest areas and look out points to take photographs and absorb the view. If you drive clockwise, you get to be on the edge of the road closest to the lake, so you get the best view from the inside track. As I mentioned before, words don’t do it justice, it’s pretty spectacular.
On our return journey, we headed about 190 miles for the coast via Klamath California. The highlight of that drive was the incredible Siskiyou /Klamath National Redwood Forests. We stayed at the Historic Requa Inn, overlooking the Klamath River. The people at the inn are very friendly, and love cats! There are some interesting geographical landforms in that area due to the build up of silt from the river as it winds downstream.
Heading down to Fort Bragg was another 200 mile journey, but the weather had cooled off a lot by this point, so we just enjoyed the mist and clouds. Again, the coastal landforms were interesting; especially the eroded arches. We stayed at the North Cliff Hotel, and took a trip to Mendocino, which was lovely.
You probably know the rest; a 207 mile journey back home to San Jose a lot of it on the coast! At that point, we had decided to avoid driving through San Francisco, as we could feel the onset of the 4th July rush, and were determined to avoid it!!!

Hundred Hours: Why an MBA Thesis is a Bad Idea

June 21, 2013

At my business school, students end their MBA curriculum with a Capstone class. This class is intended to “bring it all together” and hence its moniker. It is a special pedagogical experience for student and teacher and one that should not be bypassed. It’s always good to consolidate one’s knowledge and have the pieces fall in place.

Every now and then a student prefers to opt out of this end game experience, preferring the Thesis option that may be taken in lieu of their Capstone class. This is a mistake. Unless a student is highly motivated, knows exactly what he wants to do for a thesis, and has already undertaken at least a hundred hours of exploration of the subject matter of the thesis, it is a plain waste of time. And the opportunity cost is high, because the learning from the Capstone class is priceless. Yes, Capstone is a lot of work, and it is tempting at the end of the long haul towards a graduate degree to want to give oneself a break. But especially in this case, it’s a mistake. The trade-off is not even close, unless some really special conditions are pre-existent for the Thesis. Such as

  • The student knows exactly the question to ask in the thesis. The Question is the Thesis. When a student comes to me and requests I offer up suggestions for a topic, I let him/her know that I do not want ownership of the project, which is what happens if it’s my question. It has to be the student’s. Else it’s a no go. Much the learning comes from finding the question to ask in the thesis. Without that it’s literally half-baked. And a half-baked anything is not palatable.
  • A good question is simple, yet rigorously derived, unique, and poignant. As Helen Hunt noted – “The best movies have one sentence that they are exploring, a thesis, something people can argue about over dinner afterward.”
  • As noted above, at least a hundred hours of thinking and exploring should have gone into the Thesis idea already, some of it in formulating the question of course. Without that no one (student or professor) can be confident that the question is a good one. And a good answer to a bad question is even worse than no answer at all (or a bad answer to a good question). As a senior colleague once said to me – “if something is not worth doing, it’s not worth doing well.”
  • The student should be prepared to work on the Thesis over two quarters. Rushing it through in one quarter just leads to nonsense. If a student asks me to supervise a thesis I would like to see a two-quarter plan, else I decline.
  • The student should be prepared to sell the Thesis idea to the professor. For this you need passion, which comes from a hundred hours of immersion. Thesis supervision for a professor entails quite a few hours, with no additional compensation. It’s almost like keeping office hours for an extra course when not actually going to class to teach it. So the student needs to ask – “What’s in my idea for the Professor?” – there should be a possible follow-on research paper, or something new that the professor will learn from it. A good thesis will hit a symbiotic sweet spot.

Sounds harsh? Not really. And I’ve learnt this the hard way. Without these conditions, it’s a chore for both sides of the Thesis, student and professor. At the end of it one wishes to have taken the Capstone twice over! As Manfred Eigen said – “Sometimes a scream is better than a thesis.”

Wither MOOCs? Different Degrees of Education

June 19, 2013

In April this year, the philosophy department at San Jose State
University sent a well-reasoned letter to law professor Michael Sandel
at Harvard, explaining to him why they did not need his online justice
course imposed on their students by the management of the California
State System. Being ordered to feed your children someone else’s
cooking is an affront, to put it mildly.

The debate about traditional pedagogical delivery and the new kid on
the block — online education — is alive, and the SJSU faculty went
to battle, against the online upstart, and their administrative
overlords. Universities are run by academics (or we professors like to
think so), but state systems are run by bureaucrats. Conflict in our
education system has been engineered in, sadly. Academics are never
paid enough to often realize costs are an issue, and bureaucrats only
care about costs. The truth is, the fight over online education is
being fought on a battleground of costs. It would be sad if this were
the only thing that mattered.

Like two cars driving in their lanes and obeying the rules of the
road, massively open online courses (MOOCs) and bricks and mortar
(BAM) education have been motoring down the education highway,
glancing nervously at each other but not meeting in open conflict. The
presence of a cost speedbump has eventually resulted in fenders
touching and sparked a conflagration. Road rage is immminent, and the
SJSU folks were extremely calm. So far so good.

But MOOCs do bring several benefits to the table, the first and most
compelling being the democratization of education, cost-driven or
not. For the millions of folks with no access to good education (and I
emphasize good, there is plenty of poor quality education out there)
MOOCs offer a way to learn from great teachers when students have no
access to even poor ones. Admission to today’s U.S. college system is
tilted in favor of families with resources to groom their kids for the
application process, let alone the ability to pay for college
eventually. Second, MOOCs globalize education by making it possible
for anyone in the world to sign up and excel. And if they do well,
they are offered jobs in markets they would never have access to.
Employers are more concerned that people with degrees actually have
the skills their colleges certify, and the openness of the MOOC system
ensures skills. In some ways MOOC testing has become a quasi entrance
exam to the job market. It betrays the lie in academia of real
education, where it should be harder to graduate than to get in to
play the game.

Third, MOOCs offer the much-touted “flipping the classroom and home”
phenomenon, where lecturing occurs offline and exercises (homework)
and discussion can take place in class. This is a model that has a
positive feel to it, but research and evidence of advantageous
learning outcomes remains to be provided. Fourth, MOOCs will enable
good professors to reach thousands of students, and put pressure on
faculty who teach poorly and do no research to shape up or ship
out. And that would be great. This would alleviate one of the biggest
costs of the tenure system, i.e., checked-out, expensive
professors. Fifth, almost every university is facing a classroom
shortage, i.e., insufficient plant and equipment. A hybrid MOOC-BAM
approach will relieve this capacity constraint. If half the curriculum
is pushed online, say, we could enroll up to twice the number of
students, while reducing fees. That’s a win-win.

What’s not to like? If education were a production process, isn’t
increasing output and cutting costs exactly what MOOCs do? But do they
deliver a better learning approach, and are the students that come out
of this channel better educated? In other words, should we also strive
to deliver higher quality graduates, while working on the costs? My
personal preference is that these criteria be lexicographically
ordered: quality first, cost second. If we don’t worry about quality
we’ll end up with too many people with degrees, but no education. And
quality education is not only about quality teachers, but also about
quality environments (and I don’t mean manicured lawns, stone
buildings, and expensive football stadiums). The Culture of The
University is a huge part of it all. We want not only a trained
population, but also a learned one.

BAM education has its advantages too. First, and very important,
students are directly connected to faculty doing research. I counsel
parents who ask me to help their children choose between teaching and
research schools, and I always say that if your kid is going to learn
how to really think at college, the best way is to do research, and
that is much easier to do if you are at a research university. It’s
also the reason why I think the production of knowledge (research) and
the communication of knowledge (teaching) should co-exist in
educational institutions. It’s why we don’t see a collection of
community colleges and Bell Labs like outfits rather than research

Second, quality education does not have to cost as much as it does,
the community college system and colleges in other countries manage to
deliver an effective and good product at low cost to the student, and
the subsidies are well worth it. I just finished a semester-long
motorcycle mechanics class at the CC in San Francisco, and I learned a
huge lot, in a very hands-on, refreshing, and confidence building
manner. Separating education from unrelated university overheads will
cure the system. We pay way too much for the “country club” feel of
colleges and for expensive sports teams. It’s a chicken and egg issue,
the sports teams bring in alumni donations, which then go to support
these teams to bring in more revenue. Somewhere along the way, the
point of education is lost, and money does not go towards teaching and
research. To it’s credit the MOOC model is a wake up call to these
paradoxes of our system.

Third, the BAM model is one of the last bastions against
commoditization of our entire lives. When everything is becoming “one
size fits all”, we cannot afford to let education go in that
direction. As all of us with children know, no two siblings are alike,
and they need different handling. And our kids do prosper in a system
where they choose (voluntarily or otherwise) what sort of place of
learning system they fit in to. No doubt some may fit the MOOC model
better and that’s what they should adopt.

What will we end up with? Will MOOCs kill off all the universities?
Will the MOOCs of brand name universities (such as Harvard and MIT’s
edX) decimate smaller, less branded schools? Will we end up with some
hybrid system? Will we get fully online degrees and will they be
valued differently from BAM ones? How will we measure the quality of
education in this new world?

I worry whether I will have a job in ten years. Will my profession go
the way of airline travel agents and physical book stores? It’s hard
to say, but I suppose I’ve had a good education, and it’s trained me
to learn, so I’ll go off and do something else with my skills. What
will decide how this plays out? Quality (I would like to see the
system with the higher quality product win) or cost? For the economics of MOOCs, see this interesting graphic.

But I think there is another important determinant than quality and
cost. It is the human factor, and it will save the BAM model from
extinction. The primary reason why brick and mortar universities will
survive is that most people are unable to muster up the discipline to
work through a sufficient number of meaningful online courses, without
structure, discourse, and no transmission of perspective. Those who
can are disciplined and talented human beings for whom going to
college does not really matter. Those who cannot really do need to go
to college, but they seem to get degrees but often no
education. Herein lies the wasteful and expensive paradox of modern
day undergraduate education. If modern day college education
eventually flounders it will be because it collapses under the weight
of its own costs, not because of a technology driven paradigm
shift. But if it survives, it will be because of a deep human need for
a learned society engendered in a place of deep and higher learning –
The University. Hope springs eternal.

So you want to do a PhD?

December 26, 2012

Here is a brief extract of comments I made in an interview in Singapore this year. 

The first piece of advice I give to PhD students is to not be in a big rush to get the data and do something with it. You have to have a really good question first. Otherwise, no matter how good your data is, your work is not going to be interesting and you will not publish a good paper. 

The second thing is, you should always theoretically analyse the question completely first because that exercise will take you to the right empirical specification for the data work that you want to do. There should be a good theory because once you have a theory, then you can derive a setup that tells you that if this assumption holds, then you should see this in the data, etc. “No theory, no paper.” 

Another piece of advice is not to be in a big rush to find a topic because you are going to be stuck with it for the duration of the PhD and possibly a few more years. You do not want sub-optimally close on something that you will not enjoy. 

Additional advice is to start writing right away, do not wait; the process of writing helps you structure your thoughts better, structure your theory better, structure the empirical work better. Of course you should always be ready to throw things away. 

You should write early rather than late and keep writing. That is part of the life of a researcher; writing a lot is a good thing and throwing away a lot is also a good thing. And you know, you keep learning through that process, and that is important.

You should not expect to learn everything you need to know in a PhD programme; that is never going to happen. There were countless things I learnt after my PhD because I needed to solve a problem, I needed a technique that I did not know and I had to learn it for myself; you should be learning how to learn instead of thinking that everything is going to happen in the PhD programme. That is the purpose of the programme.

Research Environments

December 12, 2011

What makes a research environment in a business school thrive? Not the same stuff that makes firms profitable or breeds winning sports teams. It would be too easy.

The good and the bad: Consider the following scenarios I see played out at campuses across the country: Just recently someone asked me – “Why do you come in to work when you are tenured?” He looked at me as if I was irrational and that I should be away somewhere lying on a beach or making heaps of money by starting a company. And again, another junior professor inquires of a tenured one – “Why are you in today, when you are not teaching?” I have noticed at many universities that junior faculty do not go to seminars as much as senior ones. Is this a permanent cultural shift in academia? How come junior faculty are not being shepherded by seniors as they used to be in the old days? Why are there so many empty offices, and so little discussion in the hallways? Is this a reflection of the new generation of young faculty, or is it a reflection of norms gleaned from the senior ones?

There are warning signs, but also signs of encouragement and hope. There is more collaborative work than before, a greater number of seminars, and higher promotion standards, as the old excuses for the inability to publish are being shelved, as a few brave young faculty show us all how it can be done, even as journal space becomes scarcer and aggressively fought over.

A Categorization of Faculty: We may think of research faculty along two dimensions: (a) Activity: producing high quality knowledge, and (b) Presence: engaged with research activities. Just being research active is not enough, being present to facilitate the research environment is important. Viewed along these two dimensions gives us four types of faculty: {Active, Not Active} x {Present, Not Present}. I suppose it is easy to categorize any faculty into these four buckets. What type are you? Which bucket do you fall in?

The {Not Active, Present} category is more common than we might imagine. This is part of a natural cycle where Active faculty burn out and yet stay in the game by being Present. Indeed their experience is still valuable as they maintain standards, carry valuable institutional memory and perspectives, mentor junior faculty, and can judge work to ensure that promotion standards are not lowered. This is also often a temporary phase where folks take a pause to recharge batteries and then embark anew, often on something fresh.

The {Active, Not Present} group presents an interesting challenge. How does the organization engage them so that they are more than the sum of their own activity? Being present means fostering spillover benefits. But make no mistake, being Present is a double-edged sword, it is costly to Active faculty but also comes with benefits. Too much of it can be counterproductive. But, none of it is more surely a dead loss to the research organization and the researcher as well.

Outcomes: There are several benefits/costs of tenured faculty being Present, depending on which way you look at it. Here are some:

  1. Mentoring junior faculty.
  2. Working with junior faculty and enhancing tenured faculty Activity.
  3. Sharing the burden of ad-hoc demands on the faculty that otherwise have to be borne by the Present set.
  4. Quicker and higher quality group decisions in a face-to-face manner.
  5. Keeping tenured faculty expertise “on-line” for access by others.
  6. Better utilization of office space, a scarce resource in academia today.
  7. Demonstration effect/Building culture: setting a good example and high expectations for the next generation.
  8. More attendance at seminars to improve quality of discussion, and show a strong presence to outsiders so as to leave a good impression of the school’s scholarship quality and environment.

There are several flavors of Present. Some faculty are introverted but work intensely with a few junior faculty. Others help a broad swath of faculty or resolutely maintain standards. Some don’t come to work a lot, but come to all seminars and engage with juniors. And so on. We academics are all unique and we must each choose our own brand of Present.

Assume a highly Active and Present senior faculty. What features of the junior faculty would be evidence of the benefits of such a research environment?

  1. Higher quality and quantity of scholarship.
  2. Junior faculty who actively talk about their research with other faculty. This enhances the general awareness of what everyone is working on, and brings benefits of several faculty offering suggestions and pointing out related research. A self-exciting and perpetuating research atmosphere.
  3. Junior faculty who give their papers to others to read, actively asking for comments.
  4. Junior faculty that actively seek out engagement with practitioners, and continually ask themselves how they might influence practice.
  5. Junior faculty that actively take steps to enhance the research environment by organizing seminars, inviting special speakers, setting up labs, engaging in popularizing scholarly work that transcends mere publishing.
  6. Confident junior faculty who actively demand more resources for their research, and also generate their own means and funding.
  7. Junior faculty who work with faculty in other departments and disciplines.
  8. A greater number of submissions to top journals.

Making progress: How does a school take steps towards this idealized environment? First, measurement. Take stock by classifying existing faculty into the four bucket model. This is easy for department chairs to do. Place each faculty person on the 2×2 {Active, Not Active} x {Present, Not Present} grid. What is a reasonable number in each bucket?

Second, self-realization. Discuss the grid with the faculty and ask them to think about where they are and where they might want to be.

Third, incentivize movements into the {Active, Present} bucket. Use a coarse system of raises. Faculty in the {Not Active, Not Present} bucket get no raise, and in fact should be moved into shared offices. Faculty in the {Active, Present} bucket should get three times the raise of those on the {Active, Not Present} and {Not Active, Present} buckets, recognizing that being Active and Present is worth more than just the sum of the two dimensions.

Fourth, raise salaries to market so that departments can compete in the marketplace, and raise funding to support research so that junior faculty are able to compete with others from other schools.

Lesson: There is a tide in the affairs of research schools, when taken at the flood, leads on to fortune.