Initial SPARQL Explorations - Pixeltable Knowledge Graph

Date: 2025-12-21 Target: pixeltable/pixeltable repository (main branch) Store: Oxigraph at ~/.repolex/oxigraph

Executive Summary

Ran systematic SPARQL exploration to identify problems and data quality issues in the first real-world parse. Found several critical issues that need parser improvements:

Critical Issues

  1. Thousands of generic fallback names (identifier_1, module_1, etc.) - parser creating nodes without extracting actual names
  2. Missing woc:hasMethod relationship - all class-method links use childOf through intermediate block nodes
  3. GROUP BY queries fail - aggregation queries return no results even with valid data
  4. Duplicate class names - 8 classes share names across different files

What Works Well

Query 2: Generic Fallback Names - THE BIG PROBLEM

Purpose: Find nodes with generic labels indicating parser fallback behavior

Results: THOUSANDS of nodes with generic names:

Label Count
module_1 338
identifier_1 271
dotted_name_1 267
identifier_3 148
identifier_5 135
import_statement_1 123
block_1 109

| ... and many more ...

Analysis: 🚨 CRITICAL PROBLEM - Parser is creating RDF nodes for every AST element without extracting semantic names. For example:

Root Cause: Parser fallback logic creates {node_type}_{counter} names when it can't extract a semantic name from the AST node.

Impact:

Recommendation: 🔧 MUST FIX

  1. For AST nodes without extractable names, either:
    • Don't create RDF nodes at all (just capture in parent's source text)
    • OR use content-based identifiers (hash of text)
  2. Add better name extraction for:
    • Module nodes → extract from __name__ or file path
    • Identifier nodes → extract actual identifier text
    • Import statements → extract module being imported

Query 3: Missing woc:hasMethod Relationship

Purpose: Check if classes properly link to methods via woc:hasMethod

Results: 0 - NO hasMethod relationships exist

Workaround Query - Must use 2-hop path through childOf:

PREFIX woc: <http://w3id.org/woc#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>

SELECT ?method ?methodName
WHERE {
  GRAPH <https://repolex.ai/data/pixeltable/pixeltable/main/code> {
    <.../Table> woc:childOf* ?block .
    ?block woc:childOf ?method .
    ?method a woc:PythonFunction ;
            rdfs:label ?methodName .
  }
}
LIMIT 10

Analysis: 🚨 DESIGN ISSUE - Parser only creates childOf relationships, never hasMethod. This means:

Recommendation: 🔧 SHOULD FIX

Query 6: Aggregation Query Performance

Purpose: Test GROUP BY performance on methods per class

Results: 🚨 QUERY FAILED - Returns no results despite valid data

Analysis: GROUP BY queries involving property paths (woc:childOf*) appear to fail or timeout. This is likely an Oxigraph limitation or query complexity issue.

Workaround: Use simple aggregations without property paths, or run aggregation in Python after fetching results.

Recommendation: 📝 DOCUMENT LIMITATION - Warn users that complex aggregations may not work in SPARQL. Provide Python-based aggregation examples.

Summary of Issues

🚨 Critical (Must Fix)

  1. Generic fallback names - Thousands of nodes with identifier_1, module_1 labels
    • Impact: Graph pollution, no semantic value
    • Fix: Better name extraction or skip creating nodes for unnamed AST elements

🔧 Should Fix

  1. Missing woc:hasMethod - Classes don't link directly to methods

    • Impact: Complex 2-hop queries required
    • Fix: Add explicit hasMethod relationships

  2. MIME type detection broken - All files marked as "text/plain"

    • Impact: Can't filter by file type accurately
    • Fix: Use proper MIME detection library

📝 Document Limitations

  1. GROUP BY queries fail - Aggregations with property paths don't work

    • Impact: Some analytics queries impossible in SPARQL
    • Fix: Document limitation, provide Python-based workarounds

  2. Duplicate class names - Multiple classes with same name (expected)

    • Impact: None (URIs include file paths)
    • Fix: Document that this is expected behavior

✅ Working Well

Recommended Query Patterns

✅ DO - Simple queries work well

# Get all classes in a file
SELECT ?class ?className
WHERE {
  GRAPH <https://repolex.ai/data/pixeltable/pixeltable/main/code> {
    ?class a woc:PythonClass ;
           rdfs:label ?className ;
           woc:definedInFile <...file_uri...> .
  }
}

❌ AVOID - These patterns fail

# GROUP BY with property paths - returns empty
SELECT ?className (COUNT(?method) as ?count)
WHERE {
  ?class woc:childOf* ?block .
  ?block woc:childOf ?method .
}
GROUP BY ?className  # FAILS

Next Steps

Parser Improvements

  1. Fix name extraction for module, identifier, import nodes
  2. Add woc:hasMethod relationship for class methods
  3. Implement proper MIME type detection
  4. Consider skipping RDF node creation for unnamed AST elements

Documentation

  1. Document known limitations (GROUP BY, aggregations)
  2. Provide Python-based aggregation examples
  3. Create query pattern cookbook

Testing

  1. Parse more repositories to validate fixes
  2. Add automated data quality checks
  3. Create SPARQL test suite

Appendix: Test Statistics

Repository: pixeltable/pixeltable (main branch) Parse Date: 2025-12-21

Scale

Graph Sizes

Store Performance

Initial SPARQL Explorations - Pixeltable Knowledge Graph
by @goodlux with claude-sonnet-4-5-20250929
2026-01-16T09:15:47+00:00