My views in this article are a result of my past trajectory and they hold zero universal truth. It is dangerous if they are thoughtlessly deemed true by others, or by the future me. A paragraph about what the title means hasn't been written yet.

---

Today I participated in an HCI study with my labmates. The study wanted to know what counts as AI research, in our perspectives, and what is good AI research, in our perspectives.

These questions are really hard to answer.

---

To answer the first question, I brought up my “who serves who” model.

I think AI research can be broadly categorized into two types: one in which AI serves X, and another in which AI is the subject under investigation. In the first type, X can be education/math/earth/health/safety, you name it, and AI tells you what objectives to search for, what representations to build and how to optimize towards objectives (stealing Yonatan's answer a bit). As for the second type, one can generally think of it as the science of AI, and as a field it really needs to borrow insights from (thus “being served by”) other sciences, including CS/statistics/math/physics/psychology/linguistics, you name it.

From “the science of AI”, you can either enlarge or shrink the scope. If you shrink the scope, you will have “the science of deep learning”, “the science of LLMs”, etc. If you enlarge the scope, you may drop the term “artificial” and inquire about the nature of intelligence. This is where I am heading. To elaborate, the big questions I am chasing are What's hard to learn? Why is it hard? To what extent "learning = computation" is true? To what extent "cognition = computation" is true? These questions require us to drop "artificial" because they involve the study of abstract principles of learning/cognition independent of the physical substrate by which it is realized. This view actually falls in line with the original intents of founders of AI and Cogsci, but for some reason is only held by a niche community nowadays (as least this is what I perceive, also noted here). AI and Cogsci originated from the same place but have gone their separate ways. One factor accounting for their divergence is "which physical substrate is latched onto". Cogsci people study the brain, i.e. the biological implementation, whereas people study machines, i.e. the digital implementation.

---

To answer the second question, I said I can possibly say something about what's good 'research', but I don't know what's good 'AI research' because this would circle back to the question of what AI is.

To explain what’s good research in my view, I brought up my model of knowledge measurement.

I think the totality of knowledge (of an individual or some party of people) is a mass, which is an integration over three dimensions: base area (breadth), height (depth) and density (resolution). So, good research is one that builds upon a large knowledge mass and tries to extend its frontiers.

---

There are millions of maps of the research landscape and there is no standard one. Nor do we need to consolidate a million into one.

Perceiving the geometry of research landscape is a highly personal process. But it is hard to overstate its importance.

• It supports your judgment of relevance, i.e. distance between two research threads, within or across fields; the ratio of max{within-field distance} to min{cross-field distance}.
• It supports your sense of fertility of the land, i.e. “what and how much is potentially knowable out there”

You develop and refine this perception by constructing a map of the landscape as you navigate, with an ultimate goal of finding some unclaimed but cute land for yourself to excavate/discover/build something.

Others can navigate the same regions as you do but they may develop totally different-looking maps. It's okay and it's fairly likely. The reason is both objective and subjective.

Objective reason: research landscape is folded, multi-faceted, with holes and caves hidden here and there, just as the maximally rich ecological landscapes of the Earth.

Subjective reason: People exhibit biases when judging the distance. Regions that you have scrutinized feel larger for you than for someone else. If someone else has not gone through the prolonged exploration process as you did, then they will easily miss the entire maze of interwoven tunnels and halls which did a good job in inflating your perception of distance and space. Also, people tend to judge the distance between a random point to a “landmark” shorter than it actually is. Everyone has their personal labeling of landmarks. Thus, when you encounter someone in a conversation who keeps bringing up topics you cannot see the relevance, they are not trying to correct your map of the research landscape. They are disclosing important information about themselves —— they are telling you what their landmarks are.

Hence, “what counts as AI research” is an impossible question. I can give a “canonical answer” but any non-canonical answer I could give would only be valid within myself. In fact, as you grow your knowledge mass, at some point you may realize that it is inevitable to go beyond what is “canonically” viewed as AI and step onto some “exotic” land. If this happens, it does not indicate that you navigated this in a wrong manner. It indicates that you are not “boxed up”.

As I tried to think harder about this question, I ended up with “the distributional semantics approach” of defining a research area 😂. One's research area is defined by who they share a border with. Knowing who your neighbors are helps a lot with knowing where you are. But again, this is highly personal and each individual researcher/research team may differently map out the structure of relevant fields and have unique judgment of distance.

---

You are not lost. You are “not captured”.

One day a friend said they feel like they are wandering aimlessly, jumping here and there and not settling at any research topic. Obviously this makes one panic. I guess this is related to our fundamental fear of “not knowing where to go”. Evolutionarily, one might claim that “not knowing where to go” endangers survival. But this fear may not be as rooted as how it is perceived on the first sight. There is certainly a lot of room for arguments like how social expectations and the education system implant this type of fear into us. The bad feeling comes more from comparison than from within. “Not knowing where to go” is not as worse as “not knowing where to go but everybody around me seems to know”. One way to temporarily feel better is by asking a follow-up to the italic sentence: “wait, do they?”. One way to more long-lastingly feel better is by mastering the skill to function well under uncertainty. That doesn't mean you're not allowed to seek a more certain life. But you will arrive there out of seeking something you value, rather than out of uncertainty-aversion.

One should always be able to refine the skill to function under uncertainty. Paths down the following directions are endless: identifying the predictable, predicting the predictable, letting oneself appreciate the uncertain.

I think even the best scientist cannot predict discovery ahead of time. Then what's the point of sharpening your sense? There is a difference between the following two things:

1. "I know there's gem underneath! Let me dig deeper and reach it!"
2. "I know there's something underneath. I don't know if I will find apples or gummy bears or dementors or resurrection stones, but there's definitely something there!"

The point is you can always sharpen your sense to get better at (2), but nobody is able to do (1).

We have to do a round of self-introduction in our lab meetings every now and then when new members join or when we feel that it's been long since last time everybody knows what everybody else is doing. And every time, I end up saying something largely different from what I said in the previous time. I have received many questions about how I decided to switch research directions. Internally, I thought “did I ever have one? or do I always have one, which I do not know how to articulate?”. I believe it's the latter, and I'm indeed keeping looking for an articulation. For all those existing articulations that I have brought myself reasonably close to at some past moment, I don't feel that they fit me well. There is always something off. And exactly for this reason, I don't really like people tagging me based on what I “appear to do” 😂. For example, after I did WebQA, people thought I'm a multimodal person. Then when I did the skew project, people thought I'm a diffusion person.

At two occasions, I even almost tricked myself into believing that “perhaps I've found the right articulation for my research 😃”. Those occasions are:

1. When I got facinated with Anthropic's Transformer circuits thread, I thought I might be captured by the mechanistic interpretability community.
2. When I did the counting project, I thought the expressivity community is the community for me.

---

A digression

“But you will arrive there out of seeking something you value, rather than out of uncertainty-aversion.”

I would like to digress a bit and write down how the same argument applies to dealing with exhaustion. I used to hold strong opinions against the structure of master programs at CMU. Some qualities probably hold in any highly competitive and intensive universities. My negative opinions originated from how I went down a highly non-smooth path and I used to blame the environment a lot about how inhumane and impatient it is.

A digression of digression: I assume nobody is interested in hearing about my painful years but there might be somebody who wants to hear how I grew wiser thanks to those years. I do plan to write them down some time later, after I have cleared the top 100 things in my current to-do list. If you are one of my friends in real life and wants to hear about it, you can make it happen sooner by letting me know. If you are not one of my friends in real life, you can try to become one and make both (becoming my friend and hearing about how I grew) happen sooner by letting me know.

My experience has been that the “environment” (right, when I feel suffered but don't know who to blame, I tend to target at the “environment”. So here I'm talking about the “environment” as an umbrella term of many things possibly out there. It is the “my heart in my work" value imposed on us. This is a nice motto. But one can easily see an ironic flavor in the context of an overworking culture.) wants me to sprint unstoppingly and taking a rest is not really an option. If you take a rest, you may risk being eternally expelled from the game. At some point I was questioning the universe really angrily why I had to face all hard tasks all the time and there was not even moderate tasks to balance it. It felt like I was being forced to function at 100% throughput all the time and I was craving for a rest, not necessarily some really “off” time with 0% throughput, but some moderately-busy time with perhaps 75% throughput. I asked a senior person “won't it be better if I can spend a year in the industry just as a temporary 'break'?”, because completing the crazy PhD years and taking an academic job straight ahead seems impossible to me. But unfortunately the answer I got was “if you spend a year in the industry, it would be really hard for you to come back in academia”. Okay, I did not effectively process this answer because I refused to accept the reality, if anything about the reality was implied by this answer. And I had this sadness-invoking view of the PhD journey, where you are a footless bird, the subject of a poem I couldn't remember where I read from —— "There is a bird with no feet, flying across the jungles with no place to rest. It lands only once, when it is dead."

Recently I experienced a transition after which I no longer associate the footless bird with a saddening picture. I think one way to conceptualize an outcome of PhD training is that, before training, when you are thrown into a “fiery swirl”, you quickly develop mental or physical illness and you barely survive. But after training, when you are again thrown into a “fiery swirl”, you know how to keep yourself from sinking and find windows to breathe. The ultimate goal of a footless bird isn't to land at all, it is to fully embrace its status as persistently flying and be perfectly okay with it. Again, in this state, you can still rest, but you will rest because you appreciate the value of resting, rather than out of intensity-aversion.