titlebot-ng-ng/README.md

# titlebot-ng
A Matrix chat bot for Jupiter Broadcasting's "The Lunch" chat room that collects and posts title suggestions.

### From the Owner
This has gotten out of hand. I've been woking 7 days a week for a couple years now and this has been my most recent mortar between the bricks of time that I've been squeezing out of my life between sleep. Simplification needs to happen whenever possible. 

# Technical Outline

At the moment, there are 6 processes that make up the whole project and must be run simultaneously:

- scraper.py 
- post_results.py
- get_user_list.py
- get_votes.py
- jsonEndpoint/jep
- voteServer/vs

### scraper.py
What it says on the tin. Reads configured target matrix room and collects suggestions

### post_results.py
Listens for the !gettitles command and readies the votes

### get_user_list.py
Gets a list of users in the room who have more than n votes and puts them in a database so that the voting can have some degree of validation beyond random voters being allowed in.

### get_votes.py
Listens for the !getvotes command in the chat room, reads the output of the http voting, and writes the results back to the room

### jsonEndpoint/jep
Provides a json endpoint for a list of all provided votes and who voted for them

### voteServer/vs
Provides a webpage where voters can vote










## Features
- **Suggestion Collection**: Monitors chat for messages starting with `!suggest`, `!Suggest`, `!suggestion`, `!Suggestion`, `SUGGEST`, or `!sug`
- **Interactive Voting**: Posts suggestions individually with 👍 reactions for voting
- **Command System**: 
  - `!gettitles` - Triggers posting of collected suggestions
  - `!mayhem` - Generates chaotic text using Ollama AI
  - `!countvotes` - Counts upvotes on posted suggestions
  - `!getvotes` - Posts current vote results from the voting server
  - `!voteurl` - Shares the voting URL where users can cast their votes
- **Persistent Storage**: SQLite database per room with duplicate prevention
- **Acknowledgment System**: Reacts with ✅ to confirm suggestion collection
- **Web Voting Interface**: Dedicated voting server with vote validation

## Quick Start
1. Create and activate a Python virtual environment:
   ```bash
   python3 -m venv .venv
   source .venv/bin/activate
   ```

2. Install dependencies:
   ```bash
   pip install matrix-nio pyyaml requests
   ```

3. Configure the bot:
   ```bash
   cp config_example.yaml config.yaml
   # Edit config.yaml with your Matrix credentials and room ID
   ```

4. Run the bot components:
   ```bash
   # Terminal 1: Collect suggestions
   python3 scraper.py
   
   # Terminal 2: Post suggestions when requested
   python3 post_results.py
   ```

## Configuration
Create `config.yaml` based on `config_example.yaml`:

```yaml
username: "your_bot_username"
password: "your_bot_password"
room_id: "!your_room_id:homeserver.com"
homeserver: "https://matrix.org"  # Optional, defaults to matrix.org
lookback_limit: 7200  # Seconds to look back for suggestions (2 hours)
batch_size: 100       # Messages to fetch per batch
dbName: "suggestions.db"  # Optional, auto-generated if not set
vote_host: "localhost"  # Hostname for voting interface (e.g., "localhost", "example.com", "vote.mysite.xyz")
```

## Architecture
- **`room.py`**: Core `Room` class handling Matrix connection, database operations, and command processing
- **`post.py`**: `Post` class representing individual suggestions
- **`scraper.py`**: Main suggestion collection loop
- **`post_results.py`**: Suggestion posting service that waits for `!gettitles` commands
- **`get_votes.py`**: Vote results posting service that waits for `!getvotes` commands
- **`get_user_list.py`**: User list collection service for vote validation
- **`jsonEndpoint/`**: Go service providing JSON API for submissions and users (port 9080)
- **`voteServer/`**: Go web server for voting interface (port 9081)
- **`customOllama/`**: Custom Ollama model for mayhem text generation
- **`run_all.sh`**: Master script to start all services

## Database Schema
- **suggestions**: Stores collected suggestions with metadata
- **gettitles/mayhem/countvotes/getvotes/voteurl**: Command execution logs to prevent duplicates
- **last_run/last_post**: Timestamps for state management
- **users**: User information for vote validation

## Dependencies
- **matrix-nio**: Matrix client library
- **pyyaml**: Configuration file parsing
- **requests**: HTTP requests for Ollama integration
- **sqlite3**: Built-in database support

## Optional: Ollama Integration
For `!mayhem` command functionality:
1. Install [Ollama](https://ollama.ai/)
2. Build the custom model: `ollama create mad_cat_man:latest -f customOllama/Modelfile`
3. Ensure Ollama is running on `localhost:11434`

## Notes
- Each room gets its own SQLite database for suggestions
- The bot acknowledges collected suggestions with ✅ reactions
- Voting uses 👍 reactions on individual suggestion posts
- Command deduplication prevents spam and duplicate executions
- Safe database naming handles special characters in room IDs


# TODO:
- Rewrite in Rust
- Rewrite in Zig
- Rewrite in GO
<s>- Lookback is being a PITA. Needs work </s>