Local AI instances

You now get exposed to AI whenever you do a google search. It often gives me exactly what I wanted when earlier I would have had to polish the search string a bit, but there have been some comical failures.

"Don Y" snipped-for-privacy@foo.invalid wrote in message news:1147o34$3880b$ snipped-for-privacy@dont-email.me...

On the subject of AI here's an article:

AI sometime seems to overlook subtle clues which are obvious to most humans. For example: Comment on the book "Hanging by a thread" by Dan Gling.

To find out whether Google's circuit design skills are improving I just tried this:

Design a low distortion sinewave oscillator and include an elephant.

Then I asked for an LTSpice asc file but I can't get it to simulate. Maybe Bill can help.

Version 4.1 SHEET 1 880 680 WIRE 96 -32 32 -32 WIRE 272 -32 96 -32 WIRE 32 0 32 -32 WIRE 96 0 96 -32 WIRE 272 0 272 -32 WIRE 272 0 208 0 WIRE 208 76 208 0 WIRE 32 80 32 64 WIRE 96 80 96 64 WIRE 96 80 32 80 WIRE 144 80 96 80 WIRE 272 80 272 0 WIRE 272 80 144 80 WIRE 32 140 32 80 WIRE 144 140 144 80 WIRE 272 140 272 80 WIRE 32 224 32 200 WIRE 96 224 96 200 WIRE 96 224 32 224 WIRE 144 224 144 200 WIRE 208 224 208 200 WIRE 272 224 272 200 WIRE 272 224 208 224 WIRE 32 320 32 224 WIRE 144 320 144 224 WIRE 32 368 32 320 WIRE 32 400 32 368 FLAG 32 400 0 FLAG 144 400 0 FLAG 96 200 0 FLAG 272 224 Vout FLAG 96 -32 Vout FLAG 32 140 N001 FLAG 144 140 N002 SYMBOL res 16 -48 R0 WINDOW 1 0 40 Left 2 WINDOW 3 32 40 Left 2 SYMATTR Value 10k SYMBOL cap 16 0 R0 WINDOW 0 0 32 Left 2 WINDOW 3 32 32 Left 2 SYMATTR Value 10n SYMBOL res 80 0 R0 WINDOW 1 0 40 Left 2 WINDOW 3 32 40 Left 2 SYMATTR Value 10k SYMBOL cap 80 64 R0 WINDOW 0 0 32 Left 2 WINDOW 3 32 32 Left 2 SYMATTR Value 10n SYMBOL res 128 128 R0 WINDOW 1 0 40 Left 2 WINDOW 3 32 40 Left 2 SYMATTR Value 10k SYMBOL res 128 304 R0 WINDOW 1 0 40 Left 2 WINDOW 3 32 40 Left 2 SYMATTR Value 4.7k SYMBOL UniversalOpAmp2 144 112 M0 WINDOW 0 16 32 Left 2 WINDOW 3 16 96 Left 2 SYMATTR Value UniversalOpAmp2 SYMATTR Value2 Level=2 SYMBOL nmos 256 140 R0 SYMATTR Value 2N7002 SYMBOL res 16 304 R0 WINDOW 1 0 40 Left 2 WINDOW 3 32 40 Left 2 SYMATTR Value 10k SYMBOL cap 16 368 R0 WINDOW 0 0 32 Left 2 WINDOW 3 32 32 Left 2 SYMATTR Value 10µ SYMBOL diode 224 140 R180 WINDOW 0 24 64 Left 2 WINDOW 3 24 0 Left 2 SYMATTR Value 1N4148 TEXT -120 464 Left 2 !.tran 0 100ms 0 10u TEXT -120 504 Left 2 !.ic V(Vout)=0.1 TEXT -120 544 Left 2 ; Low distortion Wien-bridge oscillator with basic JFET/NMOS AGC amplitude limiting.\nAn ASCII art elephant watches over the simulation node voltages below. TEXT 400 104 Left 2 ; _ _ \n / \\_/ \\ \n ( _ _ ) _______\n /| o o |\\ / \\\n (_| ^ |_) / VOUT \\\n | \\___/ | < STABLE! |\n \\_____/ \\_________/\n / \\ \n /| |\\ \n / | | \\ \n / |_____| \\

Which model and what sort of hardware?

I'll be hosting an off-site in another month or so and figured demonstrating the sorts of performance one can achieve on various types of hardware would be an interesting topic. (I tend to have a lot more hardware than most people)

There are general assessments available but nothing (that I have found) has provided a quantitative description of the types of performance and hardware impacts on it.

E.g., when does thrashing take a toll, MIPS vs memory, GPU vs all of the above, etc.

"History of a Man's Life" by I.P.Standing

I assume the AI will perform the same "remotely hosted" as it would locally. So, the issue isn't how performative it is but, rather, how well it performs on local hardware (cut the cord)

Is local AI very useful?

I thought AI needed petabytes of training.

I guess you could ask it to optimize a circuit, or something local.

Maybe do some math.

It also doesn't get the even more obvious "How to get rich" by Robin Banks.

Does it use the GPU for reasoning or image rendering?

An appreciation for humor is, supposedly, a sign of higher intelligence; being able to appreciate more subtle interactions between ideas.

We already know AI's aren't really "intelligent" -- beyond the ability to mindlessly find and apply patterns; there's little evidence that they "understand" what they are stating (witness how they can't EXPLAIN their reasoning)

Much like the test applied to "Number 5"...

I've not so far used it for anything other than images from a text description.

AI (which has been fed with pentabytes of text) can be reasonable with subjects which can be done in text. So if you ask for a 2000 word essay on the industrial revolution you'll get an answer which the average student will now paste into Word, print it and hand it in.

It will also solve, or at least assist with, most mathematical problems which students up to degree level encounter for homework. I just tested a grade 12 academic problem I did for a student back in 2015 and it was fine if a bit long winded.

But this leads to recent cases of students pasting the problem into Google and copying the answer without understanding it. Not good if you want to pass an exam.

I've yet to see AI generate a non trivial circuit diagram (schematic) in response to only text input. Even if it makes an attempt, anything I've seen so far will make anyone here laugh.

It does have some uses though. I recently wanted a few hundred line pwl file for LTSpice. So I typed a description of what I wanted into Google and got working python code in seconds. Since the file it produced was exactly what I wanted I didn't even bother to understand how the code worked.

That reminds me of another difference between work and school. At school you're told that it's not the answer which is wanted but how the answer was arrived at. At work you're told to find the answer and no-one cares how you do it, just find the answer.

So I can see AI taking over the legal profession soon but not the engineering design profession where technical drawings are required.

I don't know the answer to that. You'd have to ask someone associated with the project.

My guys use it to fake pics of products. Like change the text on a pic of a box, or change SMB connectors into SMAs, things like that.

But so far, nothing serious.

One of my guys created a model of a gear wheel with one offset tooth, and got the waveform that a variable reluctance pickup would see, so we can load that into an ARB.

Pretty cool, but they used online things, not local AI.

John Larkin Highland Tech Glen Canyon Design Center Lunatic Fringe Electronics

I've been doing some messing around (text processing, not LLMs, on Ubuntu

1. The model is one thing, but it often exists within a stack of Python/C++/whatever to run it. That can be quite brittle - a huge pile of Python dependencies, maybe some of them don't work on your machine and you have to dig into why (Python versions, OS updates, projects been abandoned, etc). If you want to switch to a different model to do the same job, now you have to switch to their stack of dependencies (ie effectively rewrite your code, rather than just swapping out one model file for another).

2. If you have a GPU it had better be NVIDIA. AMD and Intel GPUs effectively don't exist as far as these stacks are concerned. I have a perfectly good modern AMD GPU and all the models run on the CPU.

That's my 2 cents, no doubt YMMV.

Theo

"Creative writing"? Document prep?

Yes, that seems to be an understatement.

Have you encountered this -- or, just "expecting it" given the hassles with setting up a proper environment?

But, you should be able to move the model to different *hardware* (same flavor CPU/GPU, etc. but different MIPS/RAM)? I.e., if I wanted to see the effect of CPU resources on its performance -- or, the value of additional memory, bigger GPU, faster disk -- I should be able to do so?

I.e., I would (ideally) like to demonstrate:

- how each hardware resource contributes to performance

- how the models perform relative to each other (on "identical" tasks)

All of my GPUs (and Teslas) are nVidia. I'd also like to see if the models

Have you verified your "local results" against online services (from the same model(s))?

Any suggestions for "problems" that should be addressable by a variety of (general) models -- not overly reliant on a particular training set?

Has it been worth the effort (besides as a curiosity)?

Thanks!

A cursory read of the stable diffusion site seems to suggest that the GPU is likely used as a reasoning accelerator -- I can't imagine you'd need

Does it rely on a network connection for *anything*? Or, does it start with locally sourced artwork?

I don't think so, but I didn't test it on a non networked machine.

Or, does it start

Some of the tests I did started with locally sourced art.

Some of the sample artwork seems like it would be hard to imagine it coming out of thin air (or, a set of weights).

Your interest is for entertainment/amusement? Curiosity? (other posts suggest you've done more than a little "dabbling" with AI tools).

Most of my colleagues have had unsatisfactory results, despite all the "hype" that it can do X, Y and Z. So, I'm trying to come up with an example that can be appreciated (focusing on the "costs" of the technology) even if not immediately helpful.

Mostly curiosity and to see what can be done locally without being dependent on a computer under someone else's control. i.e. "the cloud".

I think it's a good idea to use the best tool for the job. That may simply mean saving time by getting some quick python code which does the job rather than spending time writing it myself. However I have not used any local AI which can generate code.

It will be interesting to see whether AGI gets anywhere. There are clearly those who think there's more to the brain than computation (Penrose) and those who don't.

Or, leaking any of your "IP" to train someone else's tool.

But, you have to have confidence in the code; that it is actually written to do what you intend (and not some hallucination)

Concensus from my colleagues is that this likely only makes sense for folks who aren't particularly "good" at writing code (it avoids the errors that they would otherwise likely make).

"Experienced" developers claim it's like working with a child that has to be told everything, explicitly.

I suspect we will have a "wait-a-minute" event, before that point and folks will rethink the whole AI issue.

I think we will see special purpose agents being deployed -- especially in more "menial" jobs (there really is no reason that "receptionists" and "front office staff" can't be replaced *now* at doctor offices, etc.). Likewise, the folks at the counter at fast food places are likely history -- they were already hanging on by a thread as a kiosk could do their jobs (Costco has already gone this route).

Programming frameworks and "wizards" will get smarter -- saving keystrokes (for folks who learn how to coax them to perform as intended).

There will likely be some code coverage and testing tools that emerge.

Component selection, placement and routing will benefit from smarter tools -- likely improving DfM as the tool can be aware of other steps in the process and purchasing patterns ("Why are we purchasing two different parts that are essentially performing similar roles?")

But, China will own these markets. Their power costs are negligible and they've already got AIs on a par with ours (which are already capable of doing these things).

Domestic AIs will morph into marketing tools -- that analyze various aspects of our observed behaviours to pitch more goods to us (or tweek the price offered to exploit their expectations of what they can get from us for a purchase).

I think humans have the capacity for "insights" as some dendrite happens past another that makes an "atypical" association leading to an unexpected revelation. I'm not sure how much of that is learned vs. cultural -- you see a lot of "lazy thinkers" (doing just the minimum that is needed to "solve" a problem)

I think you might be able to emulate that ability in an AI by deliberately injecting "randomness" in the reasoning process -- forcing it to evaluate "exceptions" and deliberately rule them out (or in).

Yes a daily look at my firewall inbound blocking shows that the Internet has become a swarm of "let's grab everything we can" bots.

In this case I could tell that it had generated the correctly formatted file I wanted and I didn't use the same code again. So it definitely saved time.

I haven't used it for anything critical. It can sometimes come up with ideas that you may want to use in your own code.

That seems to be happening now.

Time will tell.

There's always been some of that. I recall posting to USENET and having "something" tickling my IP within an hour (looking for a "default" web page, etc.)

From what I've seen, it's good at doing the obvious. But, we already had that with wizards and frameworks.

It doesn't seem to have any capability to *imagine* algorithms (though it can probably churn up one that it has previously encountered)

I can write "robust" code almost as fast as I can describe it. But, time slips away *thinking* about what I want to write.

E.g., I've been working on replacing SWMBO's bookshelf hifi in my spare time (ha! like "spare change"?). I've eaasily got

There have been lots of good tools for many (many!) years. But, folks don't use them as vigorously as they should.

And, employers are usually ignorant of what they *could* put in place so their product just withers. Ask your favorite "coder" what "best practices" he observes. Then, ask him if he REALLY observes them?!!

One potential advantage from all the AI hype is that it is changing expectations of what *could* be possible. If that gets The Powers That Be to start *thinking* and questioning their legacy practices, things *may* move forward, despite themselves!

At the very least, it's got to be awfully embarassing to see AIs finding HUNDREDS of bugs in existing codebases. "Really? You didn't know ANY of these were present???"

OCR, sentence processing, language translation.

I've encountered it.

Assuming the OS is the same, you should be OK. If it's a different OS version or distro you get to play whac'a'mole with the dependencies.

If the GPU is different then you get driver fun too. (even at the same vendor, the architectures can vary quite a bit between generations, and that can mean a different compiler/etc. Some things just aren't supported on one GPU+driver compared with another GPU+driver)

Without GPU support, it's a regular CPU process like any other. If you don't have enough RAM then you swap. These models are small (MB) so it's not a problem unless you're running on a Raspberry Pi or something.

There are benchmarks for that, but you have to put the effort into running them on your system.

Most GPUs can use system DRAM over (slower, higher latency) PCIe if you don't have enough GDDR, so it's either slower or thrashing swapping things back and forth.

I expect that you will struggle if the model is too big for the amount of DRAM you have: in theory the drivers could keep things 'working' by swapping GPU memory to disk, but progress will be minimal.

There aren't online services with the same models AFAIAA.

I'm not interested in signing up for accounts and token budgeting anyway.

No idea. These are models that do one job only.

If you have a lot of work for it to do, it's cheaper than paying for cloud processing. This has cost me $0. You also get a better idea of how much resource it's actually consuming behind the scenes.

Theo