titlebot-ng-ng/README.md

titlebot-ng

A Matrix chat bot for Jupiter Broadcasting's "The Lunch" shows "The Lunch" and "Linux Unplugged" chat room that collects and posts title suggestions.

From the Owner

This has gotten out of hand. I've been woking 7 days a week for a couple years now and this has been my most recent mortar between the bricks of time that I've been squeezing out of my life between sleep. Simplification needs to happen whenever possible.

Technical Outline

At the moment, there are 6 processes that make up the whole project and must be run simultaneously:

• scraper.py
• post_results.py
• get_user_list.py
• get_votes.py
• jsonEndpoint/jep
• voteServer/vs

scraper.py

What it says on the tin. Reads configured target matrix room and collects suggestions

post_results.py

Listens for the !gettitles command and readies the votes

get_user_list.py

Gets a list of users in the room who have more than n votes and puts them in a database so that the voting can have some degree of validation beyond random voters being allowed in.

get_votes.py

Listens for the !getvotes command in the chat room, reads the output of the http voting, and writes the results back to the room

jsonEndpoint/jep

Provides a json endpoint for a list of all provided votes and who voted for them

voteServer/vs

Provides a webpage where voters can vote

Features

• Suggestion Collection: Monitors chat for messages starting with !suggest, !Suggest, !suggestion, !Suggestion, SUGGEST, or !sug
• Interactive Voting: Posts suggestions individually with 👍 reactions for voting
• Command System:
  • !gettitles - Triggers posting of collected suggestions
  • !mayhem - Generates chaotic text using Ollama AI
  • !countvotes - Counts upvotes on posted suggestions
  • !getvotes - Posts current vote results from the voting server
  • !voteurl - Shares the voting URL where users can cast their votes
• Persistent Storage: SQLite database per room with duplicate prevention
• Acknowledgment System: Reacts with ✅ to confirm suggestion collection
• Web Voting Interface: Dedicated voting server with vote validation

Quick Start

1. Create and activate a Python virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate
   

2. Install dependencies:

   pip install matrix-nio pyyaml requests
   

3. Configure the bot:

   cp config_example.yaml config.yaml
   # Edit config.yaml with your Matrix credentials and room ID
   

4. Run the bot components:

   # Terminal 1: Collect suggestions
   python3 scraper.py
   
   # Terminal 2: Post suggestions when requested
   python3 post_results.py
   

Configuration

Create config.yaml based on config_example.yaml:

username: "your_bot_username"
password: "your_bot_password"
room_id: "!your_room_id:homeserver.com"
homeserver: "https://matrix.org"  # Optional, defaults to matrix.org
lookback_limit: 7200  # Seconds to look back for suggestions (2 hours)
batch_size: 100       # Messages to fetch per batch
dbName: "suggestions.db"  # Optional, auto-generated if not set
vote_host: "localhost"  # Hostname for voting interface (e.g., "localhost", "example.com", "vote.mysite.xyz")

Architecture

• room.py: Core Room class handling Matrix connection, database operations, and command processing
• post.py: Post class representing individual suggestions
• scraper.py: Main suggestion collection loop
• post_results.py: Suggestion posting service that waits for !gettitles commands
• get_votes.py: Vote results posting service that waits for !getvotes commands
• get_user_list.py: User list collection service for vote validation
• jsonEndpoint/: Go service providing JSON API for submissions and users (port 9080)
• voteServer/: Go web server for voting interface (port 9081)
• customOllama/: Custom Ollama model for mayhem text generation
• run_all.sh: Master script to start all services

Database Schema

• suggestions: Stores collected suggestions with metadata
• gettitles/mayhem/countvotes/getvotes/voteurl: Command execution logs to prevent duplicates
• last_run/last_post: Timestamps for state management
• users: User information for vote validation

Dependencies

• matrix-nio: Matrix client library
• pyyaml: Configuration file parsing
• requests: HTTP requests for Ollama integration
• sqlite3: Built-in database support

Optional: Ollama Integration

For !mayhem command functionality:

1. Install Ollama
2. Build the custom model: ollama create mad_cat_man:latest -f customOllama/Modelfile
3. Ensure Ollama is running on localhost:11434

Notes

• Each room gets its own SQLite database for suggestions
• The bot acknowledges collected suggestions with ✅ reactions
• Voting uses 👍 reactions on individual suggestion posts
• Command deduplication prevents spam and duplicate executions
• Safe database naming handles special characters in room IDs

TODO:

• Rewrite in Rust
• Rewrite in Zig
• Rewrite in GO - Lookback is being a PITA. Needs work