titlebot-ng-ng/README.md

titlebot-ng

A Matrix chat bot for Jupiter Broadcasting's "The Lunch" and "Linux Unplugged" chat rooms that collects and posts title suggestions with web-based voting.

From the Owner

This has gotten out of hand. I've been working 7 days a week for a couple years now and this has been my most recent mortar between the bricks of time that I've been squeezing out of my life between sleep. Simplification needs to happen whenever possible.

Technical Outline

The project is a mix of Python (Matrix bot services) and Go (voting backend). Each room gets its own instance of every Python service, so the number of running processes scales with the number of configured rooms.

Services:

• scraper.py - Reads configured target Matrix room and collects suggestions
• post_results.py - Listens for the !gettitles command and posts suggestions for voting
• get_user_list.py - Collects the list of users in the room (with post counts and last activity) for vote validation
• get_votes.py - Listens for the !getvotes command and posts vote results to the room
• jsonEndpoint/ - Go service (port 9080) exposing JSON endpoints for submissions and users
• voteServer/ - Go service (port 9081) serving the web voting interface

Everything is launched by run_all.sh, which auto-installs Go, creates the Python venv, discovers room configs, builds the Go services, and starts one instance of each Python service per room.

Features

• Suggestion Collection: Monitors chat for messages starting with !suggest, !Suggest, !suggestion, !Suggestion, SUGGEST, or !sug
• Interactive Voting: Posts suggestions individually with 👍 reactions for voting
• Web Voting Interface: Dedicated voting server with validation (exact display-name match + activity window)
• Multi-Room Support: One config file per room under configs/, with URL slugs for routing
• Command System:
  • !gettitles - Triggers posting of collected suggestions
  • !mayhem - Generates chaotic text using Ollama AI
  • !countvotes - Counts 👍 upvotes on posted suggestions
  • !getvotes - Posts current results from the voting server
  • !voteurl - Shares the voting URL where users can cast their votes
• Persistent Storage: Per-room SQLite databases with duplicate prevention
• Acknowledgment System: Reacts with ✅ to confirm suggestion collection
• Command Deduplication: All commands are logged to prevent duplicate executions

Quick Start

1. Clone the repo and copy the example config for each room you want to monitor:

   cp configs/config_example.yaml configs/config.yaml
   # Edit configs/config.yaml with your Matrix credentials and room ID
   

2. Run everything with a single command (installs Go and creates the venv if missing):

   ./run_all.sh
   

   Or run the components individually:

   python3 -m venv .venv && source .venv/bin/activate
   pip install matrix-nio pyyaml requests
   cd jsonEndpoint && go build -o jep . && ./jep
   cd voteServer && go build -o vs . && ./vs
   

Configuration

One YAML file per room in configs/. Example (configs/config_example.yaml):

username: "magbot"
password: "your_bot_password"
room_id: "!your_room_id:homeserver.com"  # Matrix room to monitor
lookback_seconds: 86400  # 24 hours
lookback_limit: 400      # Max messages to look back when scraping
dump_mode: False         # Dump raw messages when scraping (debugging)
dbName: "my_db_name"     # Optional, auto-derived from room_id if empty
vote_host: "localhost"   # Hostname for voting interface (e.g., "example.com", "vote.mysite.xyz")
# vote_path: "my-room-slug"  # Optional URL slug; auto-derived from room name if unset
vote_activity_hours: 72  # Hours a user must have been active within to be eligible to vote

The bot resolves each room's voting page to a URL slug (vote_path config override, otherwise derived from the room name/alias) and persists the mapping to room_slugs.json, which the Go services read for routing.

Architecture

• room.py: Core Room class handling Matrix connection, database operations, and command processing
• post.py: Post class representing individual suggestions
• scraper.py: Main suggestion collection loop
• post_results.py: Suggestion posting service that waits for !gettitles commands
• get_votes.py: Vote results posting service that waits for !getvotes commands
• get_user_list.py: User list collection service for vote validation
• jsonEndpoint/: Go service providing JSON API for submissions and users (port 9080)
• voteServer/: Go web server for voting interface (port 9081)
• customOllama/: Custom Ollama model for mayhem text generation
• run_all.sh: Master script to start all services

Database Schema

Per room (suffixed with a sanitized room ID):

• suggestions.db: Collected suggestions with metadata (is_posted, event_id)
• users.db: User info (display_name, num_posts, last_active) for vote validation, plus a fetch_log of collection runs
• votes.db: Web votes cast via the voting interface

Shared:

• last_run.db / last_post.db: Timestamps for state management
• Command log tables (gettitles, mayhem, countvotes, getvotes, voteurl) live inside each room's suggestions database and prevent duplicate executions

Dependencies

• Python: matrix-nio, pyyaml, requests, sqlite3 (built-in)
• Go: github.com/mattn/go-sqlite3, gopkg.in/yaml.v2

Optional: Ollama Integration

For !mayhem command functionality:

1. Install Ollama
2. Build the custom model: ollama create mad_cat_man:latest -f customOllama/Modelfile
3. Ensure Ollama is running on localhost:11434

Notes

• Each room gets its own SQLite database for suggestions
• The bot acknowledges collected suggestions with ✅ reactions
• Voting uses 👍 reactions on individual suggestion posts and a web voting interface
• Command deduplication prevents spam and duplicate executions
• Safe database naming handles special characters in room IDs
• Web votes are tied to a user's exact Matrix display name; voters must have been active in the room within vote_activity_hours of the most recent room activity

TODO:

• Rewrite in Rust
• Rewrite in Zig
• Rewrite in GO - Lookback is being a PITA. Needs work