Library File Structure Map

LIBRARY FILE STRUCTURE MAP

Version: 1.0 Date: 18 December 2025 Author: Francesco Pelosio Purpose: Human-readable guide to library folder structure

---

📋 OVERVIEW

The Library is BlackTrails’ knowledge base system. It stores all documentation as Markdown files with YAML frontmatter, which are then synced to the database (library.documents) for RAG (Retrieval Augmented Generation) semantic search.

Key Concepts:

• Files = Source of Truth: All docs live as .md files in /library/
• Sync Script: npm run sync-library pushes files → database
• Public vs Private: Public docs sync to DB and are API-accessible; Private docs stay local-only
• i18n Support: Folder structure separates languages (/it/, /en/)
• Categories: Documents organized by domain (filosofia, progetti, algoritmi, legal, strategie, finanza)

---

📁 COMPLETE FILE STRUCTURE

E:\BLACKTRAILS-PLATFORM\library\
│
├── README.md                    # Library documentation (system guide)
│
├── public/                      # Public documents (synced to DB, API-accessible)
│   │
│   ├── it/                      # Italian documents
│   │   ├── filosofia/           # Philosophy & Vision
│   │   │   ├── rhama.md         # Birth of Rhama (poetry concept)
│   │   │   ├── rhama-paradigm.md # Rhama Paradigm (technical philosophy)
│   │   │   └── manifesto.md     # BlackTrails Manifesto
│   │   │
│   │   ├── progetti/            # Projects & Case Studies
│   │   │   └── (empty - future content)
│   │   │
│   │   ├── algoritmi/           # Algorithms & Technical Docs
│   │   │   └── (empty - future IN-1 algorithm doc)
│   │   │
│   │   └── legal/               # Legal Documents
│   │       └── (empty - future ToS, Privacy, Cookie Policy)
│   │
│   └── en/                      # English documents
│       ├── philosophy/          # Philosophy & Vision (English)
│       │   ├── rhama.md         # Birth of Rhama (EN translation)
│       │   └── rhama-paradigm.md # Rhama Paradigm (EN translation)
│       │
│       ├── projects/            # Projects & Case Studies (English)
│       │   └── (empty - future content)
│       │
│       ├── algorithms/          # Algorithms & Technical Docs (English)
│       │   └── (empty - future content)
│       │
│       └── legal/               # Legal Documents (English)
│           └── (empty - future content)
│
└── private/                     # Private documents (NOT synced, local-only)
    ├── strategie/               # Business Strategies (internal)
    │   └── (internal strategic docs)
    │
    └── finanza/                 # Financial Documents (internal)
        └── (budgets, invoices, financial planning)

---

📚 CATEGORY EXPLANATIONS

1. filosofia / philosophy 🌲

What it contains: Core vision, philosophy, and cultural documents

Current Files:

• rhama.md - Personal story about the birth of Rhama (poetry concept for IN-1)
• rhama-paradigm.md - Technical philosophy behind Rhama (how AI poetry works)
• manifesto.md - BlackTrails company manifesto (vision, principles, products)

Purpose:

• Explains why BlackTrails exists
• Defines our approach to AI (amplification, not replacement)
• Documents the emotional/cultural foundation of products like IN-1

Who reads it:

• Team members (onboarding)
• Potential partners/investors
• Public users curious about our philosophy

---

2. progetti / projects 📦

What it contains: Case studies, completed projects, client work documentation

Current Files: (empty - future content)

Planned Content:

• IN-1 launch case study
• Elements booking system implementation
• Client project retrospectives
• Technical deep-dives on specific features

Purpose:

• Showcase real-world implementations
• Document lessons learned
• Serve as portfolio/credibility

Who reads it:

• Potential clients
• Team for reference
• Public developers learning from our approach

---

3. algoritmi / algorithms 🧮

What it contains: Technical documentation for algorithms and systems

Current Files: (empty - future IN-1 algorithm doc)

Planned Content:

• ALMA V2 (IN-1 conversation algorithm) - 5-turn dialogue system
• Emotion detection logic
• Haiku generation pipeline
• Tree planting API integration
• RAG semantic search implementation

Purpose:

• Document how our systems work technically
• Enable team members to understand/modify algorithms
• Transparency for users (open methodology)

Who reads it:

• Developers (internal/external)
• Technical partners
• Researchers interested in our approach

---

4. legal / legal ⚖️

What it contains: Legal documents (terms, privacy, compliance)

Current Files: (empty - future content)

Planned Content:

• Terms of Service
• Privacy Policy
• Cookie Policy
• GDPR compliance documentation
• Data processing agreements

Purpose:

• Legal compliance
• User transparency
• Protect company and users

Who reads it:

• Users (mandatory for account signup)
• Legal team
• Auditors/regulators

---

5. strategie / strategies 🎯 (PRIVATE)

What it contains: Internal business strategies, competitive analysis, roadmaps

Current Files: (internal - not disclosed)

Example Content:

• Product roadmaps (next 6-12 months)
• Market positioning strategies
• Competitive analysis
• Growth plans
• Partnership strategies

Purpose:

• Internal planning
• Strategic alignment
• Confidential decision-making

Who reads it:

• Francesco (founder)
• Core team
• Strategic advisors (under NDA)

⚠️ NOT SYNCED: These files do NOT go to the database and are NOT accessible via API.

---

6. finanza / finance 💰 (PRIVATE)

What it contains: Financial documents, budgets, invoices, cost tracking

Current Files: (internal - not disclosed)

Example Content:

• Monthly budgets
• Invoice archives
• Cost breakdowns (infrastructure, services)
• Revenue tracking
• Tax documentation

Purpose:

• Financial planning
• Cost control
• Tax compliance
• Audit trail

Who reads it:

• Francesco (founder)
• Accountant
• Tax advisor

⚠️ NOT SYNCED: These files do NOT go to the database and are NOT accessible via API.

---

📝 DOCUMENT FORMAT (FRONTMATTER)

Every .md file in /public/ uses this structure:

---
title: "Document Title"
slug: "it/filosofia/rhama"
language: "it"
translation_id: "rhama"
category: "filosofia"
public: true
author: "Francesco Pelosio"
last_updated: 2025-12-18
tags: ["tag1", "tag2", "tag3"]
---

# Markdown Content

Your document content here...

Field Explanations:

Field	Required	Description
title	✅ Yes	Human-readable title
slug	✅ Yes	URL-friendly identifier (e.g., it/filosofia/rhama)
language	✅ Yes	Language code (it, en, es, fr, de)
translation_id	⚠️ Optional	Links translated versions (e.g., all “rhama” docs have same ID)
category	✅ Yes	Category (filosofia, progetti, algoritmi, legal, strategie, finanza)
public	✅ Yes	true = API-accessible, false = private
author	✅ Yes	Document author name
last_updated	✅ Yes	Date of last update (YYYY-MM-DD)
tags	⚠️ Optional	Array of tags for search/filtering
description	⚠️ Optional	Short description for SEO
featured	⚠️ Optional	true to highlight document

---

🔄 SYNC WORKFLOW

How Files Become Database Records

1. Write File: Create .md file in /library/public/it/filosofia/manifesto.md

2. Add Frontmatter: Include all required YAML fields

3. Run Sync:

   npm run sync-library

4. Script Actions:

   • Scans all .md files in library/public/
   • Parses frontmatter + content
   • Calculates MD5 hash of content
   • Checks library.documents table
   • If document exists (by slug):
     • Compare hash
     • If changed → UPDATE record
     • If unchanged → SKIP
   • If document doesn’t exist:
     • INSERT new record

5. Database Result: New row in library.documents table

Example Sync Output

Syncing library/public/ → library.documents...

  + it/filosofia/manifesto.md (CREATED - new document)
  ✓ it/filosofia/rhama.md (UPDATED - hash changed)
  - en/philosophy/rhama.md (SKIPPED - no changes)

Summary:
  - Scanned: 3 files
  - Created: 1
  - Updated: 1
  - Skipped: 1

✓ Sync complete

---

🗄️ DATABASE MAPPING

After sync, files map to database like this:

File: library/public/it/filosofia/rhama.md

Maps to: library.documents table row:

Column	Value	Source
id	1	Auto-generated (SERIAL)
slug	it/filosofia/rhama	Frontmatter: slug
title	La Nascita di Rhama	Frontmatter: title
category	filosofia	Frontmatter: category
content	# RHAMA\n\n## La Nascita...	Full markdown content
content_hash	a1b2c3d4e5f6...	MD5 hash (change detection)
public	true	Frontmatter: public
author	Francesco Pelosio	Frontmatter: author
tags	{filosofia, rhama, poesia}	Frontmatter: tags (array)
metadata	{"dedicated_to": "A tutte le mamme..."}	Extra frontmatter fields (JSONB)
language	it	Frontmatter: language
translation_id	rhama	Frontmatter: translation_id
last_updated	2025-12-18 10:59:00	Frontmatter: last_updated
created_at	2025-12-18 09:00:00	Auto-generated on INSERT

---

🔍 FINDING TRANSLATED DOCUMENTS

Example: Find all versions of “rhama” document

SELECT
    slug,
    title,
    language
FROM library.documents
WHERE translation_id = 'rhama'
ORDER BY language;

Result:

it/filosofia/rhama          | La Nascita di Rhama        | it
en/philosophy/rhama         | The Birth of Rhama         | en

---

🌐 i18n STRATEGY

Folder Structure

• Language-First: /public/it/ vs /public/en/
• Same Categories: Both languages have same folder structure (filosofia, progetti, etc.)
• Translation Linking: Use translation_id in frontmatter to link versions

Example: Italian + English

Italian File: /library/public/it/filosofia/rhama.md

---
slug: "it/filosofia/rhama"
language: "it"
translation_id: "rhama"
---

English File: /library/public/en/philosophy/rhama.md

---
slug: "en/philosophy/rhama"
language: "en"
translation_id: "rhama"
---

Query for Both:

SELECT * FROM library.documents WHERE translation_id = 'rhama';

---

🔐 SECURITY & ACCESS

Public Documents (/library/public/)

• ✅ Synced to database
• ✅ API-accessible via /api/library/documents
• ✅ Used in RAG semantic search
• ✅ Can be embedded in apps (IN-1, docs site, etc.)

Example API Request:

GET https://in-1.ai/api/library/documents?category=filosofia&language=it

Private Documents (/library/private/)

• ❌ NOT synced to database
• ❌ NOT API-accessible
• ❌ NOT searchable via RAG
• ✅ Local filesystem only
• ✅ Can be git-ignored (optional)

Use Cases:

• Internal strategic planning
• Financial documents
• Confidential client notes
• Personal brainstorming

---

🛠️ MAINTENANCE COMMANDS

Create New Document

# Create file
touch library/public/it/progetti/nuovo-progetto.md

# Add frontmatter (manually or via script)
# ...write content...

# Sync to database
npm run sync-library

Update Existing Document

# Edit file
nano library/public/it/filosofia/manifesto.md

# Update `last_updated` field in frontmatter
# Save file

# Sync (script detects hash change and updates DB)
npm run sync-library

Delete Document

# Remove file
rm library/public/it/progetti/old-project.md

# Sync (script can mark as deleted or remove from DB)
npm run sync-library

# Optional: Clean up orphaned embeddings
npm run cleanup-embeddings

List All Synced Documents

SELECT
    slug,
    title,
    category,
    language,
    last_updated
FROM library.documents
WHERE public = true
ORDER BY category, language, title;

---

📊 CURRENT STATISTICS

Files Count (As of 18 Dec 2025)

Location	Count	Size
/public/it/filosofia/	3 files	~8 KB
/public/en/philosophy/	2 files	~5 KB
/public/it/progetti/	0 files	0 KB
/public/en/projects/	0 files	0 KB
/public/it/algoritmi/	0 files	0 KB
/public/en/algorithms/	0 files	0 KB
/public/it/legal/	0 files	0 KB
/public/en/legal/	0 files	0 KB
/private/strategie/	Private	Private
/private/finanza/	Private	Private

Total Public Documents: 5 markdown files (~13 KB)

Documents in Database

SELECT
    category,
    language,
    COUNT(*) as doc_count
FROM library.documents
WHERE public = true
GROUP BY category, language
ORDER BY category, language;

Expected Result:

filosofia  | it | 3
filosofia  | en | 2

---

🚀 FUTURE PLANS

Short-Term (Next 30 Days)

• Add IN-1 algorithm documentation (algoritmi/in1-algoritmo.md)
• Create legal documents (ToS, Privacy, Cookie Policy)
• Add first project case study

Medium-Term (Next 90 Days)

• Full i18n support (Spanish, French, German)
• Automatic embedding generation on sync
• API endpoint for document suggestions (RAG-powered)
• Version history (track changes over time)

Long-Term (Next 6 Months)

• Collaborative editing (multi-author workflow)
• Automated translation via Claude
• Document analytics (view counts, search queries)
• Integration with docs.blacktrails.io (Starlight)

---

📞 SUPPORT

Library System Owner: Francesco Pelosio Email: francesco.pelosio@gmail.com Documentation: https://docs.blacktrails.io/private/area-x/infrastructure/ Database: Neon.tech (patient-queen-89181819)

---

End of Library Map 🌲📚

“Every document is a seed. Every sync is growth. Every search is discovery.”