Computation Is Not Thought

Computation Is Not Thought

All the talk of reasoning models, agents and LLM's that 'think' may inadvertently set a dangerous precedent; one in which society accepts that AI models are capable of thought, a uniquely human trait.

Familiar Footing.

As humans who witness technology permeate through society and become ingrained in our daily lives, most of us overlook the effect it has on subsequent generations who will grow up as natives in a landscape with that technology already infused in it. This risks a portion of the population having a blind spot to some of the ramifications of technology.

For example, a high percentage of children today would likely meet the threshold for addiction when it comes to digital dependency, and how it shapes priorities, decisions, actions and behaviour, yet strategies to counter it are sparse. This likely stems from the iPhone's twenty-year dominance as an unregulated cognitive extension. Adults today intuitively understand limits and restrictions because our existence predates the iPhone, but the younger generations have been affected to a greater extent.

We potentially face a similar problem with LLM's. When technology that computes is used as a substitute for thinking, it has several detrimental effects:

  • Humans reduce their thinking
  • Generated content becomes ubiquitous which lowers overall standards
  • The capacity to think atrophies.
  • Inertia wins more often
  • Aspects of memory change

In the world before cell phones, people remembered dozens of phone numbers with ease. That 'skill', and all that it enabled, has silently been lost to time. Similarly, the effort we are willing to make has changed with the abundance of instantaneous information. This has a halo effect on individual behaviour; if we can't find something, we settle for an alternative almost immediately. Copy/paste has carved out grey matter from our collective cognition. As the following article from 2019 alluded to, our thinking may be inversely related to our ability to compute.

AEON IDEAS

How Much Can We Afford to Forget, If We Train Machines to Remember?

Civilisations evolve through strategic forgetting of once-vital life skills. But can machines do all our remembering? An exploration of how AI memory partners may be changing what knowledge we retain versus outsource.

April 8, 2019 Read more

Understanding The Difference

Computation is a mechanistic process which is substrate-agnostic and objective, in that it produces outputs from given inputs. Thought, on the other hand, is a cognitive phenomenon, embedded and contextual while being intentional and semantic. It involves a self-referential layer which machines do not have.

Thinking requires identity since it encapsulates multiple levels of continuous judgments.

Cognitive load-bearing is an integral part of intellectual development. Habits formed by technological convenience leave us predisposed to lighter burdens. Where once we might sit with a difficult challenge, allowing ideas to percolate and develop through sustained contemplation and progressive iteration, we now reflexively reach for our devices at the first hint of mental effort or uncertainty; It's easier.

This leads to an insidious pursuit of convenience and instant gratification, fueled by dopamine-hunting interface design on social platforms, compulsive scrolling habits and misplaced priorities. The result leaves us misguided and weakened; with an eroded capacity to sustain focus, think independently and critically, and make sound decisions. This corruption starts at the personal level and cascades through groups, corporations, and institutions

Aspect Computation Thought
Syntax vs. Semantics Operates purely on formal symbols (syntax). Involves semantic understanding (meaning) and situational context.
Determinism Follows explicit rules; given the same input, always the same output. Although it can use rules, thought often includes intuition, heuristics, or even "non‐algorithmic" leaps.
Consciousness No requirement of awareness or subjective experience. Typically involves an awareness of oneself processing ideas (metacognition).
Creativity & Flexibility Can be highly innovative within defined parameters and rules, but operates within the framework and objectives given to it. Draws from abstract, unrelated domains to transcend existing frameworks.
Embodiment Abstract process tied to whatever hardware implements it. Deeply influenced by bodily states, emotions, and sensory feedback.
Intentionality A computation doesn't care what it's computing. Thoughts carry imagination, curiosity, priorities and intentions.
Context Sensitivity Context must be encoded explicitly; otherwise, no "background understanding." Draws on situational cues, long‐term memory, and broader world knowledge seamlessly.

LLMs perform large-scale statistical inference over learned patterns.
An answer matches your prompt's pattern to similar contexts encountered during training, then retrieves the next-token that statistically fits best. This points to a massive underlying web of learned parameter values and not an explicitly embedded ontology.

Their non-deterministic outputs arise from introduced randomness which can be controlled through temperature, top_k and top_p settings. This is the realm of computation which draws from a probability distribution computed via matrix multiplications.

Dynamic Models with web access or continuous learning can retrieve real-time information via web search or updated databases. They may have the ability to learn or correct themselves, but this depends on how information is weighed leading to whole host of ethical quandaries. Since most LLMs are fundamentally limited to patterns present in their training corpus, they can't genuinely encounter something genuinely novel and integrate it the way humans do. When humans encounter a new concept, we can:

  • Connect it to embodied experience
  • Relate it to emotional or sensory memories
  • Instantaneously develop bias or affinity based on a multitude of personal factors, and opinions stemming from our perspective, and ultimately identity
  • Anticipate future implications based on identity and goals
  • Modify our entire conceptual framework to accommodate it

LLMs have no qualia and can only interpolate within the conceptual space they've already seen. This means their 'creativity' is more accurately defined as innovation within fixed bounds and rules.

If we conflate computing and thinking, we might erroneously lean on computational solutions for problems that require abstract thinking that only humans are capable of. Or, we might undervalue human cognitive contributions that can't be replicated computationally, which has an extraordinarily high opportunity cost.

Anomaly’s Stance: Artistic Intuition Over Programmed Logic

Our position prioritizes Human Agency while recognizing that Artifical Intelligence represents a massive opportunity to bridge gaps in society.

Instead of leveraging the technology to amplify our own outputs, the trend to replace thinking is rising, perhaps led by the proliferation of Generative AI tools which prioritise quantity and speed. This leaves a vacuum for tools that prioritise quality and a design space to explore for retaining creative control.

Introducing Paradox: an AI-augmented writing studio.

We believe writing is an art and must be practiced with intentionality.
While processes vary, ours starts with a thought, a question or an idea. The development journey requires time - to reflect, research, deliberate and synthesize - evolving through many spirals of refinement.
Inspiration, information and insights can come from anywhere at anytime and need to be captured. Removing friction while retaining context and staying in flow provides focus, clarity and momentum which drastically shortens the thinking-to-publishing loop, all while retaining creative control.

Paradox was created for our own internal use to write, to think, and to write to think.

Features:

  • Preserves Cognitive Sovereignty: By removing direct conversation with the AI, users are forced to articulate their thoughts first.
  • Visual Separation of Domains: The split-screen separates human thought from machine computation while maintaining connection between the two; a link which enables collaboration and continuity.
  • Version Spine: A visual map to navigate and depict the evolution of ideas, this serves a functional time machine by managing the iterative nature of writing and thought development.
  • Floating Capture Window: Persistent note-taking interface for capturing fleeting thoughts while browsing, researching, or working away from the main writing interface
  • AI as Editor, Not Author: The compute window serves as a sophisticated editorial partner—analyzing structure, proofreading text, suggesting stylistic improvements, and refining tone—rather than generating original content. This distinction positions AI as a precision tool that improves human ability and productivity.

LLMs are perfidious teachers, but make for instructors par excellence.
Devoid of genuine artistry, they can nonetheless provide personalized insights and facilitate deepening the execution of our own unique creative signatures.