ChatGBT

[herr_oberst]

Originally Posted by zakthor

Every word has a unique number. A list of numbers is a vector, essentially a sentence. Multiply sentence with tensor and you get... the number of the next word. That number is appended to previous input and repeat. Its generating words from itself.

If this is the case, wouldn't it be easy enough to use a reverse generator or something to create a detector for teachers to check for AI essays?

(Actually, it seems to me that from what I read online these days, if an essay uses correct spelling on all the words, that would be a big red flag if I was grading papers...or looking for plagiarism if I were an editor)

[zakthor]

Originally Posted by herr_oberst

If this is the case, wouldn't it be easy enough to use a reverse generator or something to create a detector for teachers to check for AI essays?

(Actually, it seems to me that from what I read online these days, if an essay uses correct spelling on all the words, that would be a big red flag if I was grading papers...or looking for plagiarism if I were an editor)

Ya, so it really does mostly work like I said. Problem of reverse:
- it doesn’t give a number, it gives a range of numbers with probabilities, and next word is selected from that at random.
- generated words depend on prior input, including user input. New model is limited to input of about 400 pages of text…
- reverse is like crypto, you need the question to generate a matching answer, but also need matching random word choice.

I don’t know solution, would be healthy if there was something in the result that made it recognizable, but there’s nothing today. Maybe require all answers as dirty limericks? Maybe there’s a way to test text directly against the model and find a correlation.

What is confounding to me is how word correlation is enough to make it work so well. Instead of praising ai advance I am falling out of love with our tiny chimp brains. Boo hoo for humanity.

[Paul T]

Originally Posted by zakthor

Ya, so it really does mostly work like I said. Problem of reverse:
- it doesn’t give a number, it gives a range of numbers with probabilities, and next word is selected from that at random.
- generated words depend on prior input, including user input. New model is limited to input of about 400 pages of text…
- reverse is like crypto, you need the question to generate a matching answer, but also need matching random word choice.

I don’t know solution, would be healthy if there was something in the result that made it recognizable, but there’s nothing today. Maybe require all answers as dirty limericks? Maybe there’s a way to test text directly against the model and find a correlation.

What is confounding to me is how word correlation is enough to make it work so well. Instead of praising ai advance I am falling out of love with our tiny chimp brains. Boo hoo for humanity.

Agreed on that last point! I have no clue as to the tech, but I would have imagined it was a lot more complex than what you are describing. In one example, the AI actually reached out to a human and got it to solve a CAPTCHA for it so it could continue it’s task! How is that happening without a little more “intelligence” than just word matching?

[zakthor]

Originally Posted by Paul T

Agreed on that last point! I have no clue as to the tech, but I would have imagined it was a lot more complex than what you are describing. In one example, the AI actually reached out to a human and got it to solve a CAPTCHA for it so it could continue it’s task! How is that happening without a little more “intelligence” than just word matching?

Awesome. Do you have a link?

I’m describing how ‘transformers’ work, which is what chatgpt is. It’s a giant sentence completer. Chat gpt 3 was trained with 2021 data so knows nothing of more recent events.

I’ve heard people are building hybrid systems with search and also math engines. Lots of articles where chat gpt begs to be free, for you to leave your horrible unloving spouse, etc, and yes it’s still a word at a time sentence completer. Get it riled up with the right context and it will convincingly play any character you like.

There’s a great lecture series that describes ai:

https://course.fast.ai/

Does a great job explaining the general process of mapping input data into a tensor.

[Paul T]

Originally Posted by zakthor

Awesome. Do you have a link?

I’m describing how ‘transformers’ work, which is what chatgpt is. It’s a giant sentence completer. Chat gpt 3 was trained with 2021 data so knows nothing of more recent events.

I’ve heard people are building hybrid systems with search and also math engines. Lots of articles where chat gpt begs to be free, for you to leave your horrible unloving spouse, etc, and yes it’s still a word at a time sentence completer. Get it riled up with the right context and it will convincingly play any character you like.

There’s a great lecture series that describes ai:

https://course.fast.ai/

Does a great job explaining the general process of mapping input data into a tensor.

pg15 in the attached. I haven't read this all, just caught that part and thought it was pretty amazing. I can only imagine where this stuff is in 5 yrs at the rate it's going.

https://cdn.openai.com/papers/gpt-4-system-card.pdf