Future Directions: Ontology Gardening & Neurosymbolic Approaches

Date: 2025-12-21 Status: Exploration & Planning Context: Two major architectural directions for repolex evolution

Direction 1: Ontology Gardening for Code

What Is It?

Ontology Gardening is an approach where schemas evolve from observed data rather than being pre-defined. Instead of forcing all code into fixed ontologies (WOC, TreeSitter), the system learns domain-specific concepts by observing patterns across a codebase.

Core Loop:

1. Observe → TreeSitter AST + docstrings/comments
2. Extract → LLM generates SPO statements about code intent
3. Learn → GLiNER2 finds entities/relations against current schema
4. Evolve → Gardener agent proposes new concepts when needed
5. Refine → Curator agent merges duplicates, creates hierarchies

Originally developed for visual knowledge graphs (octology project), where VLMs observe images and the schema evolves to accommodate new visual concepts.

How It Would Improve Repolex

Current Limitation: Repolex extracts structure (classes, functions, AST) but doesn't understand domain semantics. Querying for "authentication code" or "data validation logic" requires knowing exact class/function names. The system has no concept of what "authentication" means in YOUR codebase.

With Gardening:

1. Domain Vocabulary Discovery

• Learns codebase-specific terms from docstrings
• "This module handles user session management" → creates SessionManagement concept
• "Validates payment data before processing" → creates PaymentValidator concept
• Enables queries like: SELECT ?code WHERE { ?code :implements :Authentication }

2. Architectural Pattern Recognition

• Discovers recurring patterns: Repository, Factory, Service Layer, Adapter
• "Class UserRepository implements data access pattern" → links to Repository concept
• Query: "Show me all repositories in this codebase"

3. Intent Relationships

• Learns verbs that describe code purpose: validates, orchestrates, caches, transforms, serializes
• Maps these to actual functions: validate_payment() → :intent "validation"
• Query: "Find all validation logic across the codebase"

4. Per-Repo Schemas

• Each repository grows its own domain ontology
• pixeltable/pixeltable learns: DataPipeline, ColumnStore, MediaProcessor, VectorIndexer
• fastapi/fastapi learns: RouteHandler, DependencyInjector, SchemaValidator
• Cross-repo queries can compare: "How does authentication differ between repos X and Y?"

Hard Problems It Would Solve

1. The "Synonym Problem"

Current Issue:

• AuthService, AuthManager, AuthHandler all do similar things
• No way to know they're related without manual inspection
• Queries miss results because names vary

Gardening Solution:

• Curator agent detects semantic similarity via docstrings
• Proposes: "These three classes all implement :AuthenticationComponent"
• Creates hierarchy: AuthService rdfs:subClassOf :AuthenticationComponent
• Queries now find all three

2. The "Intent Gap"

Current Issue:

• Can find function validate_user_input() by name
• Can't find "all validation logic" if functions named differently
• check_payment(), verify_credentials(), sanitize_data() all do validation

Gardening Solution:

• LLM analyzes docstrings: "This function validates payment data"
• Extracts intent: :intent "validation"
• All validation functions discoverable regardless of naming

3. The "Subsystem Boundary Problem"

Current Issue:

• Files organized by folder structure (opaque)
• No semantic understanding of module boundaries
• Can't query "show me the payment processing subsystem"

Gardening Solution:

• Observes import patterns + docstring mentions
• Learns: {payment.py, stripe_client.py, invoice.py} all belong to :PaymentSubsystem
• Creates explicit subsystem graph

Direction 2: Neurosymbolic Approach with Embeddings

Why Add Embeddings?

Current State:

• Symbolic: RDF triples, SPARQL queries, explicit relationships
• Strengths: Precise, queryable, explainable, logical reasoning
• Weaknesses: Brittle to naming variations, can't find "similar but not identical" code

With Embeddings:

• Neurosymbolic: Combine symbolic graph + vector similarity
• Strengths: Fuzzy matching, semantic similarity, transfer learning
• New capabilities: "Find code similar to this function", "Cluster semantically related modules"

Embedding Strategies: Pushing the Envelope

1. Code Graph Embeddings (Microsoft Research)

What: Embed the graph structure itself, not just code text.

Microsoft's GraphCodeBERT / CodeT5+:

• Encodes AST structure + data flow + control flow
• Input: Code text + graph edges
• Output: Vector that captures both syntax and semantics

Novel Queries:

• "Find functions with similar control flow structure"
• "Cluster modules by architectural similarity"

2. Hyperbolic Embeddings (Poincaré, Lorentz)

What: Embed in hyperbolic space instead of Euclidean.

Why: Code has hierarchical structure (packages → modules → classes → methods). Hyperbolic space naturally represents hierarchies with logarithmic growth.

Novel Queries:

• "Find classes closest to X in the hierarchy"
• "What's the conceptual distance between these two modules?"
• Preserve tree structure: Parent-child relationships = hyperbolic distance

3. Multi-Modal Embeddings (Code + Docs + Tests)

What: Embed functions with their documentation, tests, and usage examples in shared space.

OpenAI CLIP-style for Code:

• Contrastive learning: Align code with its docstring in embedding space
• Similar: Code ↔ Test ↔ Documentation should cluster together

Novel Queries:

• "Find code that matches this plain English description"
• "Show functions where tests don't match documentation" (divergent embeddings)
• "Find usage examples semantically similar to this one"

4. Relational Embeddings (Knowledge Graph Completion)

What: Embed entities AND relationships (predicates) as vectors.

Models:

• TransE, DistMult, ComplEx (older)
• RotatE, QuatE (rotation-based)
• TuckER (tensor factorization, SOTA)

Novel Capabilities:

• Link prediction: "What dependencies are likely but not explicit?"
• Graph completion: Fill in implicit relationships
• Similarity via relation paths: Functions are similar if they have similar relation neighborhoods

Should We Add Kuzu Entity Database?

Kuzu: Fast, embedded graph database optimized for OLAP queries, not RDF.

Pros:

• Fast aggregations: GROUP BY, COUNT much faster than Oxigraph
• Property graph model: More flexible for embeddings (store vectors as properties)
• SQL-like queries: More familiar than SPARQL for some users
• Great for analytics: Code metrics, clustering, similarity search

Cons:

• Not RDF: Separate data model, have to duplicate graph
• Maintenance burden: Keep Oxigraph + Kuzu in sync
• No SPARQL: Lose standard query language, ontologies

Recommendation: Option C (Oxigraph + FAISS/Qdrant)

Why:

• Keep RDF pure: Oxigraph stays clean, standards-compliant
• Best vector performance: FAISS is insanely fast for similarity search
• Separation of concerns: Symbolic reasoning ≠ vector similarity
• Easy to swap: Can switch vector DB without touching RDF
• Proven pattern: Lots of systems use graph DB + vector DB

No Kuzu needed unless you hit serious GROUP BY performance issues in Oxigraph (which you can test first).

Next Steps

1. Prototype Gardener - Start with minimal implementation in octology, then adapt for repolex
2. Benchmark Embeddings - Test CodeBERT-small on a medium repo (~1000 functions), measure speed
3. Design Kuzu decision - Run GROUP BY benchmarks on Oxigraph, see if we actually need Kuzu
4. Clean up CLI - Implement proper command structure with click
5. Write tests - Start with ground truth for one small repo

What direction feels most exciting to you? Gardening or embeddings?