Initial Brain Dump
Local Models + Agent Idea
TTS + RAG
I want a virtual team-lead/manager that keeps track of all my on going work and consistently follows up with me and tracks my progress based on prescribed aspects.
Hierarchy:
- The Big Boss (Me : setup high level vision and goals)
- Boss Agent ( AI Agent: consistently tracks , talks to and manages me )
- Worker Me ( Me: the one who does stuff )
Pick good models or ask-for/use big cloud model api keys for planning, tool calling and coding, for TTS use a decent but local model , RAG has to be fully local book-keeping but with integrations to some type of hosted kanban board/task tracker which can be used at all levels.
High level idea:
- Pre-start: The Big-Boss setups a vision , direction and goals , what counts as progress and how to measure them and a set of valid actions to take.
- Boss Agent: Coordinates with “Worker” to get the job done, talks to the worker periodically, sets up followups, asks questions, helps unblock, motivate, shuffles things up if the worker becomes bored/burnt-out or un-productive, re-imagines the narrative but maintains the same set goals etc. Overall a great momentary steward and a guiding light to bring the big ideas to fruition. Incase of misses or delays, emphasises but also strongly asserts how this realigns expectations and outcomes with the worker, while also communicating the challenges and explaining the direction to the Big-Boss etc.
- Worker Me: Has extreme clarity on specific tasks and deadlines, able to do deep work on specific things and maximize time blocks and work through methodically while tracking and maintaining progress at all times.
There can be a weekly or bi-weekly meeting between Big-Boss and Boss Agent , while the Worker and Boss have a daily standup.
Exploring Existing Tech
Exploring whats out there lead me to https://github.com/paperclipai/paperclip , which I’m quite intrigued by! It really pushes the idea of no-human in the loop company to its current limit. Its pretty cool to experiment with but i fundamentally believe that it is AI that is in the loop and not humans. As we ascend the abstraction chain we only have the human desire left at the higher levels, as long as AI does not “want” things, we will be the only one’s “do” ing things.
I want to take an adjacent stand with my AI in the loop idea. Firstly, a company is born of a desire for something that doesn’t yet exists, which currently is an innate human attribute. Secondly, a lot of what a company is, is its people, even with AI, i see companies as stacks with interleaving humans and AIs. Sometime AI manages humans and other times the humans the AI, quite fractal’ly all the way down. This again is fundamentally different from how paperclip thinks about it where you just ask for things to be done and get out of the way letting the AI cook, and as the dotto mentions in a recent video when an existing company tries to adopt it, humans are already there, how do we maximize in that setting?
That brings me to this current idea, while the current experiment im doing of a boss-manager-worker might not be at the right abstraction level, it still feels like the simplest singular interleaving yet useful case (at least personally). Hence worth pursuing, in fact i even plan on using paperclip to build this out if possible.
Also the current Human-AI-Human (HAH) hierarchy can be used in a wide variety of settings, personally speaking i was hoping to use this as my personal manager/task-master to help me prfioritize, track, work on my set of learning goals in TEL. But this can also be purposed to act like a middle-manager for any setting even with different human being the big-boss and a another one being the worker, providing both a top-level board member abstraction like paperclip while also streaming lining existing humans productivity, don’t you think?