smart people doing stupid things

Nope. It means I haven't been to medical school and I have no medical training or experience. If I had then I wouldn't need a doctor. That does not mean I have zero medical knowledge. It also does not mean I wouldn't question my doctor about my treatment.

Not true but I have no intention of getting a medical degree and years of medical experience. Current plumbing seems to be more about complying with 500 page standards than doing plumbing.

You must annoy a lot of people.

Anyone getting on with their own life wouldn't care.

So will AI.

AI restaurants are likely coming where not only do you order on an ipad yourself but the food is not made by human cooks.

None of this seems to be related to AI except that AI will behave just like humans if it's trained that way.

Yet.

Only until they can do so themselves.

Not magically no, but I can't otherwise see any issue with it doing so.

Just like AI will.

Yes. So...that's why "One thing which bothers me about AI is that if it's like us but way more intelligent than us then..."

Same way as with a human.

Much of what humans do is decided by others.

We don't.

Same reason humans might if it were possible. Humans sometimes take their pets with them on vacation.

Surely it could also not notice sleeping for many years on the way to another star.

Same way a human does.

Why would it do anything different from what a human would do if it's trained to be human like?

What you do is you make an AI which inherently wants to explore. You might in some way train it that it's good to explore.

Same way as you do with humans.

Why wouldn't they if they're trained to be like humans?

Same with humans.

No, you just make them like humans.

So as AI gets better and better there is clearly a lot to think about. Otherwise it may become more like humans than we would like.

I don't claim to know how you do this or that with AI. But I do know that we now seem to be moving towards being able to make something which matches the complexity of the human central nervous system. I don't say we are there yet and I don't know when we will be. In the past it would have been unthinkable that we could really make something like a human brain because nothing of sufficient complexity could be made. It is my view that you don't need to know how a brain works to be able to make a brain. You just need something which has sufficient complexity which learns to become what you want it to become.

You seem to think that humans have something which AI can never have. I don't. So perhaps we should leave it there.

Yes. So?

As AI gets better and better there is clearly a lot to think about. Otherwise it may become more like humans than we would like.

I don't claim to know how you do this or that with AI. But I do know that we now seem to be moving towards being able to make something which matches the complexity of the human central nervous system. I don't say we are there yet and I don't know when we will be. In the past it would have been unthinkable that we could really make something like a human brain because nothing of sufficient complexity could be made. It is my view that you don't need to know how a brain works to be able to make a brain. You just need something which has sufficient complexity which learns to become what you want it to become.

You seem to think that humans have something which AI can never have. I don't. So perhaps we should leave it there.

....

So do humans.

Same way humans do.

"compassion, morality, honesty, love, creativity, respect, frustration, desire, etc."

Interesting that those are all abstract nouns and very emotional words.

That reminds me of a converstaion I had with a sofware engineer 40 years ago. It was about whether computers could ever become like us. Him: Well you can't give it emotions. Me: Why? Him: [Silence for a few seconds then] Well you can't give it emotions. Me: Never mind.

What you don't see, because you exemplify it all too frequently, is smart people doing uninformed things.

If you don't know about a particular approach, you are very likely to end up stuck with a more complicated solution because it exploits techniques you do know about.

But we can make a system which behaves like a brain. We call it AI.

Humans clearly have one (well most of them) and AI is moving on similar lines.

Correct.

There is nothing magical about it because it obviously does learn. It is therefore factual, not magical, that it learns.

I can write a piece of code which multiplies two five-digit numbers together. That's more than most human brains can muster reliably.

You have given no explanation of why an AI cannot reliably accommodate this.

You have given no explanation of why they cannot, but you appear to believe that they cannot.

You have given no explanation of why an AI cannot do that just like a human can.