Computation Is Not Thought

Computation Is Not Thought

All the talk of reasoning models, agents and LLMs that 'think' may inadvertently set a dangerous precedent; one in which society accepts that AI models are capable of thought, a uniquely human trait.

Familiar Footing.

As humans who witness technology permeate through society and become ingrained in our daily lives, most of us overlook the effect it has on subsequent generations who will grow up as natives in a landscape with that technology already infused in it. This risks a portion of the population having a blind spot to some of the ramifications of technology.

For example, a high percentage of children today would likely meet the threshold for addiction when it comes to digital dependency, and how it shapes priorities, decisions, actions and behaviour, yet strategies to counter it are sparse. This likely stems from the iPhone's twenty-year dominance as an unregulated cognitive extension. Adults today intuitively understand limits and restrictions because our existence predates the iPhone, but younger generations have been affected to a greater extent.

We potentially face a similar problem with LLMs. When technology that computes is used as a substitute for thinking, it has several detrimental effects:

  • Humans reduce their thinking
  • Generated content becomes ubiquitous which lowers overall standards
  • The capacity to think atrophies.
  • Inertia wins more often
  • Aspects of memory change

In the world before cell phones, people remembered dozens of phone numbers with ease. That 'skill' and all that it enabled has silently been lost to time. Similarly, the effort we are willing to make has changed with the abundance of instantaneous information. This has a halo effect on individual behaviour; if we can't find something, we settle for an alternative almost immediately. Copy/paste has carved out grey matter from our collective cognition. As the following article from 2019 alluded to, our thinking may be inversely related to our ability to compute.

AEON IDEAS

How Much Can We Afford to Forget, If We Train Machines to Remember?

Civilisations evolve through strategic forgetting of once-vital life skills. But can machines do all our remembering? An exploration of how AI memory partners may be changing what knowledge we retain versus outsource.

April 8, 2019 Read more

Understanding The Difference

Computation is a mechanistic process which is substrate-agnostic and objective, in that it produces outputs from given inputs. Thought, on the other hand, is a cognitive phenomenon, embedded and contextual while being intentional and semantic. It involves a self-referential layer which machines do not have.

Thinking requires identity since it encapsulates multiple levels of continuous judgments.

Cognitive load-bearing is an integral part of intellectual development. Habits formed by technological convenience leave us predisposed to lighter burdens. Where once we might sit with a difficult challenge, allowing ideas to percolate and develop through sustained contemplation and progressive iteration, we now reflexively reach for our devices at the first hint of mental effort or uncertainty; it's easier.

This leads to an insidious pursuit of convenience and instant gratification, fueled by dopamine-hunting interface design on social platforms, compulsive scrolling habits and misplaced priorities. The result leaves us misguided and weakened; with an eroded capacity to sustain focus, think independently and critically, and make sound decisions. This corruption starts at the personal level and cascades through groups, corporations, and institutions

Aspect Computation Thought
Syntax vs. Semantics Operates purely on formal symbols (syntax). Involves semantic understanding (meaning) and situational context.
Determinism Follows explicit rules; given the same input, always the same output. Although it can use rules, thought often includes intuition, heuristics, or even "non‐algorithmic" leaps.
Consciousness No requirement of awareness or subjective experience. Typically involves an awareness of oneself processing ideas (metacognition).
Creativity & Flexibility Can be highly innovative within defined parameters and rules, but operates within the framework and objectives given to it. Draws from abstract, unrelated domains to transcend existing frameworks.
Embodiment Abstract process tied to whatever hardware implements it. Deeply influenced by bodily states, emotions, and sensory feedback.
Intentionality A computation doesn't care what it's computing. Thoughts carry imagination, curiosity, priorities and intentions.
Context Sensitivity Context must be encoded explicitly; otherwise, no "background understanding." Draws on situational cues, long‐term memory, and broader world knowledge seamlessly.

LLMs perform large-scale statistical inference over learned patterns.
An answer matches your prompt's pattern to similar contexts encountered during training, then retrieves the next-token that statistically fits best. This points to a massive underlying web of learned parameter values and not an explicitly embedded ontology.

Their non-deterministic outputs arise from introduced randomness which can be controlled through temperature, top_k and top_p settings. This is the realm of computation which draws from a probability distribution computed via matrix multiplications.

Dynamic models with web access or continuous learning can retrieve real-time information via web search or updated databases. They may have the ability to learn or correct themselves, but this depends on how information is weighed, leading to a whole host of ethical quandaries. Since most LLMs are fundamentally limited to patterns present in their training corpus, they can't genuinely encounter something genuinely novel and integrate it the way humans do. When humans encounter a new concept, we can:

  • Connect it to embodied experience
  • Relate it to emotional or sensory memories
  • Instantaneously develop bias or affinity based on a multitude of personal factors, and opinions stemming from our perspective, and ultimately identity
  • Anticipate future implications based on identity and goals
  • Modify our entire conceptual framework to accommodate it

LLMs have no qualia and can only interpolate within the conceptual space they've already seen. This means their 'creativity' is more accurately defined as innovation within fixed bounds and rules.

If we conflate computing and thinking, we might erroneously lean on computational solutions for problems that require abstract thinking that only humans are capable of. Or, we might undervalue human cognitive contributions that can't be replicated computationally, which has an extraordinarily high opportunity cost.

Anomaly’s Stance: Artistic Intuition Over Programmed Logic

Our position prioritizes Human Agency while recognizing that Artificial Intelligence represents a massive opportunity to bridge gaps in society.

Instead of leveraging the technology to amplify our own outputs, the trend to replace thinking is rising, perhaps led by the proliferation of Generative AI tools which prioritise quantity and speed. This leaves a vacuum for tools that prioritise quality, and a design space to explore for retaining creative control; technology that elevates, rather than erodes human intellect.

Introducing Paradox: A personal knowledge layer with AI-assistance.

Writing is an art intricately intertwined with thought, the basis of ideas and creation. Since thinking is the origin of innovation and growth, it should be nurtured and protected. With different tools at our disposal, we all follow a different creative process.

Paradox is a product spun out from our own internal use. It was created to support the way we cultivate ideas: some mind-mapping, AI-assisted chats, comparisons across conversations, visual planning, writing to add details, and prototyping on a persistent surface; a process that would have required five separate apps and a constant struggle for momentum. Few tools exist for connected workflows that enable users to retain full control over context and generations while planning for a specific objective.

Knowledge also compounds as human understanding develops and branches over time. Instead of taking thoughts to a chat interface, we needed a system that would take the things we do daily, and compound them in an additive, controllable, reusable substrate.

This is where Paradox lies: a personal, living, connected knowledge layer that grows and evolves with everyday use. Everything users do results in semantically enriched data.

Features:

  • Native Chat connected to a canvas: Chat connected to your documents, previous chats, and context connected via topic-tags. Import, highlight and distill information from your chats with ChatGPT and Claude, and inject it into new chats, or new or existing documents.
  • A canvas connected to an editor: Paradox Canvas is a visual planning and generative surface where each node is connected to Paradox Editor; a full, human-first writing app redesigned for the AI-era
  • Visual separation of domains: The split-screen separates human thought from machine computation while maintaining connection between the two; a link which enables collaboration and continuity.
  • Version Spine for Drafts: A visual map to navigate and manage the evolution of ideas, this serves a functional time machine for the continuous and branching nature of writing.
  • Multi-export: With native support for PDFs, JSON, Markdown, DOCX, HTML and TXT files, and integrations for Dropbox, Google Drive and Ghost, Paradox is a single platform covering most modern writing use cases.
  • A generative canvas for creating creating multi-channel outputs. An inertia-defeating way to create visuals, twitter threads, blog posts, LinkedIn posts and more from your writing while staying in control of the creative process.

LLMs are powerful partners for refining human thought. Combined with Paradox’s connected workflows and compounding knowledge layer, they form a system where knowledge evolves organically and ideas take shape naturally.