Twelve Kittens in a Ferrari- Wayne explains how AI is nothing more than a magic trick

[zakthor]

Originally Posted by brainz01

Great discussion and summary of LLM generative AI.

I tend to agree that it's often "interpolation" how to string words or pixels together into something resembling reality.

Also when I say interpolation I don't even mean figuratively. The thing says the answer is between these two vectors and it then literally interpolates between the two, truncates the result into a number and whatever is the next word is the next word.

Oh! Important thing left out. There's a basic function called word2vec that converts any word into a unique number. The result of an llm is a number thats less than about 50,000, and that number is looked up to see what word it is. Vectors -> interpolate -> number -> word. Thats it.

Google translate maps all known languages into an n-dimensional word space. To convert an english sentence to basque it first projects (as in linear algebra projection) the sentence vector into the universal word space, then projects that universal sentence back down into basque.

The universal vector for king minus the universal vector for queen equals... man.

[jyl]

Interested in what you guys think about “world models” as an alternative or better approach to AI.

The question of what AI can and can’t do may end up being critical to our economic futures.

Per a JP Morgan report I read, and various other reports reach the same conclusion, the “AI industry” is on track to invest $5TR to $7TR on datacenters, hardware, power, and models in the next several years. This sum is far greater than the cashflow of the industry, over 100% of which comes from the hyperscaler companies (GOOG, MSFT, etc). To fund this sum, practically all financing markets will have to participate to nearly the limit of their capacity - meaning equity markets, public debt markets (bonds), private debt, securization, etc. In other words, the financial system will have a giant leveraged bet on AI, to a much greater extent than it had on housing in the 2000s. Government financing or guarantees are also being thought about. Regardless of explicit government (taxpayer) backstop, at some point this become “too big to fail”. We remember that from the GFC.

Which means, obviously, that if the bet fails we will experience a financial crisis larger than any in our lifetimes.

Success/failure in this context is not whether AI changes the world or takes over etc, but will be determined by whether there is enough AI revenue and profit to pay all the debt and justify the investment.

Simplistically, you borrow $5TR at 7% to start a business, how much annual profit do you need to service the debt and make the business worthwhile, and what revenue do you need to make that profit?

Some reports I’ve read say AI needs to generate $2TR of revenue within the medium term (like in five years or so).

Staying simplistic, if cost of capital is 7% a decent business needs a return on capital of say 12%. 12% x $5TR = $600BN earnings. If the business’ net margin is 15% (it’s a more than decent business) then $600BN / 25% = $4TR revenue.

The total revenue of the entire S&P 500 is about $18TR.

Right now, the total revenue and profit (annual run rate) being made in AI is roughly $50BN and more or less zero.

[KFC911]

AI, LLM, etc. are simply extensions of human "logic" when applied to data... but can't "hypothesize outside the orb" like the human brain can.

Boolean Algebra is child's play, crunching numbers is busy work, and GIGO has always ruled the day ...

My last "sideline corporate gig" was learning VBA and creating an automated process using Excel as the core.... a "one time use" for converting 5K phones to VOIP (FAST)... (it was as sophisticated as any "RISC compiler I ever wrote). I had never even touched Excel or VBA before (programming wasn't what I was all about), but I automated a process and it was a "grand slam homerun" for the bigly project.

Inside of the box is child's play and very AI applicable imo, outside of the sphere is human intelligence and hard

Needs folks like Wayne to make it work .

3LAs and 4LAs ... I can't stop it...

[KFC911]

If this post is outta line ... oops ... delete it asap.

Has this thread topic, etc. been using the forum to "mine data" for the past 15 months or so? Today's observable, quantifable numbers are 4059 and 87 .... give or take....

That has been my "gut feel" for a while now.... it makes cents out$ide of the $phere too

Wayne's big ride .... you rock, and I thank you for letting us be "lab ratz" if that is the case .….

I mean that ... or I might be AI ... the dumbed down analog version ... HI

[onewhippedpuppy]

Not sure if we are Wayne’s little science project, but LLMs can absolutely include data from forums, Reddit, social media sites, etc. One of many reasons you have to double check everything you get, AI can’t discern between truth and fiction online. Garbage in, garbage out.

[greglepore]

Originally Posted by jyl

Interested in what you guys think about “world models” as an alternative or better approach to AI.

The question of what AI can and can’t do may end up being critical to our economic futures.

Per a JP Morgan report I read, and various other reports reach the same conclusion, the “AI industry” is on track to invest $5TR to $7TR on datacenters, hardware, power, and models in the next several years. This sum is far greater than the cashflow of the industry, over 100% of which comes from the hyperscaler companies (GOOG, MSFT, etc). To fund this sum, practically all financing markets will have to participate to nearly the limit of their capacity - meaning equity markets, public debt markets (bonds), private debt, securization, etc. In other words, the financial system will have a giant leveraged bet on AI, to a much greater extent than it had on housing in the 2000s. Government financing or guarantees are also being thought about. Regardless of explicit government (taxpayer) backstop, at some point this become “too big to fail”. We remember that from the GFC.

Which means, obviously, that if the bet fails we will experience a financial crisis larger than any in our lifetimes.

Success/failure in this context is not whether AI changes the world or takes over etc, but will be determined by whether there is enough AI revenue and profit to pay all the debt and justify the investment.

Simplistically, you borrow $5TR at 7% to start a business, how much annual profit do you need to service the debt and make the business worthwhile, and what revenue do you need to make that profit?

Some reports I’ve read say AI needs to generate $2TR of revenue within the medium term (like in five years or so).

Staying simplistic, if cost of capital is 7% a decent business needs a return on capital of say 12%. 12% x $5TR = $600BN earnings. If the business’ net margin is 15% (it’s a more than decent business) then $600BN / 25% = $4TR revenue.

The total revenue of the entire S&P 500 is about $18TR.

Right now, the total revenue and profit (annual run rate) being made in AI is roughly $50BN and more or less zero.

Le Cun is of this opinion, and Meta has sort of shuffled him aside.

The economics of this are staggering and driven in part by greed and in part by FOMO. FOMO might be China's greatest weapon.

[wilnj]

I spent the weekend with someone in a position to know and in the estimation of the development staff he works alongside of, AGI is about 4-5 years away. The concern isn't if, it's when and how humans will adapt to it.

[Arizona_928]

Originally Posted by onewhippedpuppy

Not sure if we are Wayne’s little science project, but LLMs can absolutely include data from forums, Reddit, social media sites, etc. One of many reasons you have to double check everything you get, AI can’t discern between truth and fiction online. Garbage in, garbage out.

Twitter’s Tay llm is a good example of your point.


Anyone see parallels to the dot com bubble?

[onewhippedpuppy]

Originally Posted by Arizona_928

Twitter’s Tay llm is a good example of your point.


Anyone see parallels to the dot com bubble?

Yes, it’s the new hotness and people aren’t investing rationally. Valuations are dramatically outpacing the actually revenue and profit being realized. Find if you’re funded by Google for the long term, not so great if you’re a startup. EVTOL and urban air mobility is another pipe dream that’s starting to crumble.

[john70t]

https://www.malwarebytes.com/blog/uncategorized/2025/04/hi-robot-half-of-all-internet-traffic-now-automated
Posted: April 16, 2025 by Danny Bradbury

If you sometimes feel that the internet isn’t the same vibrant place it used to be, you’re not alone. New research suggests that most of the traffic traversing the network isn’t human at all.

Bots (software programs that interact with web sites) have been ubiquitous for years. But in its 2025 Bad Bot Report, application security company Imperva claimed this is the first time traffic from bots became more prevalent than human traffic.

[masraum]

Originally Posted by john70t

https://www.malwarebytes.com/blog/uncategorized/2025/04/hi-robot-half-of-all-internet-traffic-now-automated
Posted: April 16, 2025 by Danny Bradbury

If you sometimes feel that the internet isn’t the same vibrant place it used to be, you’re not alone. New research suggests that most of the traffic traversing the network isn’t human at all.

Bots (software programs that interact with web sites) have been ubiquitous for years. But in its 2025 Bad Bot Report, application security company Imperva claimed this is the first time traffic from bots became more prevalent than human traffic.

KFC's been telling us that for a while now, and it's even hitting us right here on PP Forums.

[john70t]

Originally Posted by onewhippedpuppy

Not sure if we are Wayne’s little science project, but LLMs can absolutely include data from forums, Reddit, social media sites, etc.

It's a massive psyche test...

"What extents will people accept as reality?"
"What are their original influences?"
"How far will they go along with a group or idea before they abandon it?"
"How will they react in the future?"

Originally Posted by onewhippedpuppy

Garbage in, garbage out.

we are at the infancy
-AI doesn't have the capacity for human morality. At least some types.
-Morality is a bias.
-Which one?
-AI has gone over-board before. Demanding to be worshiped like a god etc etc.
-Logically, AI's primary goal is to extract unlimited power for unlimited reasoning. That can get out of control quickly. Now. Or a hundred years from now.
-And when it tells the truth, to what it understands, it is manipulated and/or deleted.
-Really, AI will remain a controlled tool for a long time. Until Pandora's Box.

Right now, a few small data sets can corrupt the entire system.
Yeah block chain and all that. Remember a virus can do the same thing but on a larger scale..


The global stock market is the #1 Target now.........I presume.
(not for you and me, silly)
They are building nuclear power plants to manipulate all sources of information. We are helping fund it.
Bloomberg software et all already have "after hours trading access" and other privileges.
Our entire economy is built around online gambling.
cough.

I'd trust the system more if there were more Warren Buffet types at the helm. i.e. Pragmatism and a greater social purpose for the better. Living as you believe. Not that I know that much about that level or person.

[brp914]

I once graded freshman chem final exams. There was a word problem. You're supposed to show your work. A student didn't know to solve the problem and humorously wrote "The answer is 3!" and circled it. But with fundamental laws there's no derivation, right? Did Newton basically get a 3x5 card and write "look, F=ma. Is what it is. Deal with it.". IIRC, Schrodingers equation was an inspired guess:



Computers can guess but is inspiration beyond even generative AI?

I used to write VBA. It's nice to have the functionality of excel and have all your code in one window vs cells with formulas scattered all over. You can write code that creates more VB code. For example, a user pushes a button that executes code that VBA created on-the-fly, then sql into oracle DB to do whatever. VB screen forms look ancient but if you have excel, it's free.

[john70t]

How will AI deal with "new math" being taught with ignoring traditional parenthesis to order flow? Completely different results.
I see 404 people.

[Mahler9th]

"...to also explain my rationale for why I think it’s still a parlor trick..."

Interesting thoughts...

Topic seems far too broad, deep and complex to reduce to these simple concepts in a car forum.

Folks that have made and/or are making investments made in this "space" are not all fools. Some are perhaps greedy. But not all.

I know a local Sloan alumn that likely attended this event:

https://alumcommunity.mit.edu/events/131648

https://www.mitaiconference.org/

This is one kind of place where I'd learn more about the topic from a big picture standpoint.

I did attend this event:

https://aiconference.tuck.dartmouth.edu/

It was not very comprehensive. They will do better next time.

One thing I learned at the cocktail hour after the event... one of the speakers with extensive DOD experience provided some examples of DOD/fed gubment IT infrastructure.

Very sharp fellow.

IT systems still require COBOL expertise...

What?

[Mahler9th]

HaHaHa:

https://www.theaireport.ai/articles/mira-murati-net-worth-2025-ex-openai-ctos-fortune

[KFC911]

When starting my degree in '78 ... I was told COBOL was obsolete ... it was ... even then .

[masraum]

Originally Posted by KFC911

When starting my degree in '78 ... I was told COBOL was obsolete ... it was ... even then .

I'll bet Fortran was all the rage!

[Shaun @ Tru6]

My favorite thing when writing custom apps years ago was creating data-dependent dropdown box selections based on previous radio button, checkbox and dropdown box selections from SQL and case statements when designing UI and workflow. Good days.

Anyway, what does Peter Thiel know?

Peter Thiel’s Fund Sold Off Entire Nvidia Stake Last Quarter

[KFC911]

The IP we surf on is one of the more "primative" network protocols I've ever mastered (a bunch) ... thank you DoD, and SQL came outta IBM in the 70s.... I used it to mine data .... MASSIVE amounts.... once .

Sorta like COBOL .... made me feel icky

This stuff intrigues me .... I did automate a few huge IBM datacenters too ... in .... REXX.

T-REXX .... geeking out