- jaypeters.dev
- Posts
- AI Training, AI Loading, and A New AI
The Weekly Variable
Plenty of work to do to get AI to do the work this week.
Plus a new way to run AI models, new AI API access, and a new AI model.
This is officially an AI newsletter again.
Topics for this week:
Upload Goals
Unsurprisingly, I did not manage to get any clips uploaded this week like I had planned to last week.
Nothing is ever as simple as it seems.
Or I just overcomplicated it.
My thinking is to do the complicated part up front.
I want an AI to identify segments in a transcript where there’s a useful or entertaining clip that could be uploaded to social media.
I’ve provided sample scripts and (publicly available) data from a few well performing YouTube channels to help guide the AI’s decisions.
The first test of this process wasn’t so great, though.
The clips it identified weren’t all that interesting, but I also only provided a single 45 minute segment of a single stream.
We’ll see what it finds with a full 12 hours stream or more.
My bet is with enough hours of recorded video, there’s bound to be at least a few gems in there.
And rather than searching through all of that video myself or trying to remember when those clips happened on my own, or even needing to build the muscle of “hey I should mark that clip” after something notable happens, instead I could have a system that automatically wades through hours of footage (transcripts) and tries to identify the best clip candidates.
Right now, this process will still result in someone needing to decide if the clip is worth posting or not, so it can’t automatically upload videos, but it can display all the clips it identified on a dashboard and let the user pick which ones are good to go.
I’m close to this process, and with each attempt, the picture becomes clearer.
But for now, I’m going to bump the goal out a little, so by the end of the month, I’m planning to have clips finally upload to YouTube.
With that finally in place, after a few weeks of daily uploads, I’ll have actual data to work with to better refine the process.
Until then, I have other options to consider below…
LM Studio
Playing around with the clip identification process and different AI models, I figured it was the perfect time to finally try out LM Studio, and save a little on additional AI API subscriptions.
Looking back, it’s hard to believe I’ve had the installer sitting in my Downloads folder, untouched, since the end of March.
I had no idea what I was missing out on because it’s an awesome tool.
I played around with oobabooga’s “text generation webui” to run AI models on my computer in the past, but that took reading an article or two and a couple YouTube videos to get working.
LM Studio only took a few minutes.
Run the installer, start up LM Studio, find an image to use (which you can search right from the Home tab), click Download, go to AI Chat, select the model you just downloaded to load, and in a few minutes you have a local model running on your laptop.
The “text generation webui” is still a great project, it got the job done, and it seems is way more customizable and extensible, but my first impression with LM Studio was very positive if you don’t need anything super fancy.
Depending on performance, and how easily these models can get the answers I’m looking for, LM Studio may be my goto solution for home builds.
Or if I need to boot up a new version of a model quickly, this may be the best way to do it.
If you’re trying to run an AI on your computer, I’d highly recommend looking at LM Studio.
Tier 3 Usage
Directly opposite to saving money on AI costs, I was running the numbers this week and trying to decide if it’s worth getting my account into Tier 4 usage for Open AI.
At the time, the only way to get API access to o1-preview was with Tier 4 or Tier 5 usage, which Tier 4 usage isn’t all that bad.
For $250 worth of API usage, you would hit Tier 4 and gain access to o1 models through the API.
Otherwise you can only use o1 through the chat interface if you’re subbed to ChatGPT plus.
I’ve been impressed with o1-preview’s performance so I was heavily debating if I want it to be the lead on finding clips for me, or maybe even flirt with the idea of it just playing social media manager.
o1-preview is exactly 100x more expensive than gpt-4p-mini at $15 per million tokens vs $.15 per million tokens, but 1 million tokens is a lot of tokens so it makes sense to eat the extra cost for a process that takes a little time to “think” but produce a better answer on average.
The only road block is that I would have to spend $250 in API costs to get there, which is a surprising amount of processing.
In all my spamming of GPT, I’m still under a total of $20 spent since I’ve had an account, which is for more than 2 years - amateur at best.
I’d have to ramp up the data processing with GPT APIs to get near that usage cap.
Luckily, this week, OpenAI lowered o1 access to Tier 3, which is only $100 in a month instead of $250.
Not a bad discount, but still a lot of API calls.
I’ll be considering if I find I need more AI processing power, but who knows, if I wait long enough, o1 models may drop down to Tier 1 usage and then I won’t have to worry about it!
I’ll only have to worry about switching to the next new model OpenAI will have introduced that caused them to make o1 easily available.
Nvidia’s First Model
We’re on a roll with AI this week so I’ll throw in my brief tangent with NVLM-1.0.
I was gearing up to work on prompting/coding for the day but I saw this article talking about Nvidia and bombshells so naturally I had to click it.
Finding that Nvidia had opensourced their own AI model that is rumored to rival GPT-4, I wanted to try it out for myself.
I’m not sure how I didn’t catch it, but after I built and booted the docker image for the model with nothing happening, I started to wonder if I could even run this AI.
Turns out, no I couldn’t, which I should have realized from the name NVLM-D-72B.
72B, being 72 billion parameters, is a lot of data, and typically won’t fit on a laptop or desktop with 64 GB of ram.
LM Studio doesn’t typically offer anything higher than a “9B” model because anything else will not fit on a local machine.
Not a complete waste of time, only 30 minutes or so wasted on building an AI I can’t run, but it was a good learning experience.
Plus if I hadn’t had been scrolling when I shouldn’t have, I wouldn’t have seen that Nvidia launched their first model, which is pretty exciting!
They’re already making the chips, why shouldn’t they be making something specially designed to run on those chips.
If you have access to a much beefier cloud infrastructure than I do and want to give NVLM 1.0 a shot, it’s waiting for you on Hugging Face.
Numeracy
Naval had a great chat with Bilaji recently, and among a number of heavy topics, there was one unexpected idea in particular that stood out to me.
Naval pointed out the difference between numeracy and literacy and how differently they are accepted.
I’ll paraphrase in 2 sentences.
If someone says they don’t know how to read, it creates an awkward social situation.
If someone says they aren’t good at math, everyone else jokes, agrees and moves on.
I might be biased because I enjoy math, but it makes me wonder where things went wrong to make numeracy so much more of a rare thing.
But rarer still is someone with numeracy and illiteracy, which probably answers that question.
You typically need to to learn to read words first before learning math, at least the more advanced math anyway.
Naval even points to this in another part of the conversation while giving advice to future founders.
Learn English because it’s become the de facto standard for nearly anything a founder would use in starting a company.
Arguably the math is less important, but it can certainly help.
Anyway, it was an interesting comparison I’d never made before, but that’s something Naval is quite good at.
His entire talk with Bilaji is worth a listen at only 51 minutes and plenty of dense ideas.
My one and only “podcast” recommendation for this week is linked below:
And that’s it for this week! Lots of AI, tweaking, loading, and failing to load. More AI talk to come!
Those are the links that stuck with me throughout the week and a glimpse into what I personally worked on.
If you want to start a newsletter like this on beehiiv and support me in the process, here’s my referral link: https://www.beehiiv.com/?via=jay-peters. Otherwise, let me know what you think at @jaypetersdotdev or email [email protected], I’d love to hear your feedback. Thanks for reading!