u/Mysterious-Rent7233

"The human brain has about 1,000 times more neurons than the mouse brain, for instance, and 13.5 times more than the macaque."

https://www.nature.com/immersive/d41586-024-03425-y/index.html

So it's hard to tease apart the returns to "just scale" versus topology.

At the end of the day, it would be impossible for Americans to feel this kind of anxiety in the same way or with the same intensity as the Canadians or Europeans who live through it. 

Donald Trump is their president. It is their neighbours who are being swept up and sent to camps. It is their freedom of speech and voting rights that are under threat. It's American women having their abortion rights curtailed.

I think it is quite possible for them to feel extreme and intense anxiety.

Trump's team negotiated CUSMA and they did so because otherwise prices would go up and exports would slump. That's still true. And further price rises just before the midterms is not a good look.

$200/month -- in 2026. How much in 2030? 2035?

So what? That has nothing to do with the claim about MoE?

But does it work? In a way that is useful?

You heard him mention "polysemantic noise". This has been known for several years:

https://transformer-circuits.pub/2023/monosemantic-features

Unfortunately, the most natural computational unit of the neural network – the neuron itself – turns out not to be a natural unit for human understanding. This is because many neurons are polysemantic: they respond to mixtures of seemingly unrelated inputs. In the vision model Inception v1, a single neuron responds to faces of cats and fronts of cars. In a small language model we discuss in this paper, a single neuron responds to a mixture of academic citations, English dialogue, HTTP requests, and Korean text. Polysemanticity makes it difficult to reason about the behavior of the network in terms of the activity of individual neurons.

In the video he says:

but if we look at layer six for example...foods um again uh completely unrelated concepts with the same feature in the same slot

This game may also interest you:

https://semantle.com/

All this is to say is that, I’ve seen what real blueberries look like when they’re added to pancakes or muffins. I know that when I bite into a baked good with real blueberries, there’s usually a gush of fruit juice, there’s texture from the skin of the blueberries, the color from the berries can stain your hands/tablecloth, etc. When I eat a blueberry Eggo, I don’t get any of that. So I know from my sensory (sensorial?) observations, that those are not real blueberries.

If you make orange juice at home, the consistency is quite different than an actual orange. And yet the main ingredient is a real orange. Or even just a dried apple. No fruit juice. Potentially no skin texture.

That's what I would think about Eggo blueberries. They've taken a real blueberry, dried it, flattened it and put it through who knows how many other machines and then put it in a waffle.

Them: "We used new tools to show how we might be able to save human lives with new analysis techniques."

You: "Not interested."

Let's imagine Reddit as a conversational interface, just to get the process kicked off. What does it look like? What value does the conversation offer?

Did you come up with it on the spot or had you heard of it before?

By bitmap do you mean simply a bit array, but packed with no wasted space?

Lean is also a highly AI-relevant data format.

https://arxiv.org/html/2505.05758v5

https://www.fields.utoronto.ca/talks/AI-Math-Neuro-Symbolic-Auto-Formalization-Lean-Joint-Embeddings

https://github.com/cmu-l3/llmlean

Just ask any AI: "Can you write me ten lines of Lean" and it will do it.

Unfortunately when I clicked the link it did not go to a specific comment. If you have a moment, could you quote it so I know what I'm looking for?

Why cant they come into your bedroom.

They can't come into my bedroom because they wouldn't want me to come into their bedroom. And society would entirely collapse if we didn't have SOME expectation of privacy and safety. Also: whether people are allowed in my bedroom is not a property of where they were born. It's a general rule for every human being on the planet other than me.

The rule of "privacy in the bedroom" is, as far as I know, universal across all societies which build separated housing.

If my nation decided to abolish this rule, on a FAIR basis, applying to EVERYONE EQUALLY then I guess I'd have to live with it, unhappily.

People have a right to have somewhere to live.

People dont have a right to live wherever they want.

Well they can't live in my bedroom. But if I want to rent to them and they want to rent, then why do we NOT have the right to conduct that transaction?

Putting aside the negative effects of immigration, theres still plenty of people wanting to emigrate that we can afford to be choosy.

You are confusing your ability to exert power over other people with their rights as human beings.

Yes, we can use laws and guns and concentration camps to "be choosy". That doesn't make it right.

Can you share a link so I can see the full context? I'd tend to agree with you and disagree with them but out of context I don't know if you've presented their argument fairly.

Ai has been trained so well on software development because all the engineers working on it are doing this themselves + every bug or line of code has already been written.

The bit you said after the "+" is simply false.

AI works well for software because:

1. Software development artifacts can often (but not always) be demonstrated to be "right" or "wrong". Things that don't compile, don't run, don't match their test cases are wrong. So you can train a machine to be good at making software that doesn't have those flaws. This software might still not meet the requirements, but a lot of the errors can be trained out of the models automatically.
2. The highest goal for these companies is to make AI which builds AI while developers sleep or nudge. They believe this is the path to super-intelligence. They believe that the first company to succeed at this will become the biggest, most important company in history. Marketing is small potatoes in comparison. Not worth much effort, relatively speaking. Their goal is, once they have AI that can make AI they will just direct the first AI to make a marketing AI and marketing will be "solved" overnight.
3. As you said, engineers know how to do engineering so its easier for them to train a model to do their own jobs than to do other people's jobs.

"every bug or line of code has already been written" -- no. Most lines of code are unique. When you combine them into functions and classes, that is even more true.

Let's say that there are two individuals. Bob and Bill.

Bob immigrated ten years ago and became a criminal. Bob broke bad.

Bill wants to immigrate now. What right do you have to tell Bill where he can or cannot rent an apartment or get a job on the basis of something a complete stranger did? How is it ethical that Bill is punished for Bob's behavior? What are other circumstances where you think that group punishments like this are ethical?

You pay for the response tokens you didn't see as well?

I don't understand why one wouldn't want a "best of both worlds" computer which can both run deterministic programs at scale and also run neural inferencing. What is to be gained by removing the deterministic infrastructure? Are they envisioning a new form of hardware which is more optimized somehow?

I'm sorry but I can't even parse your first sentence much less understand the whole comment. Can you clarify please?

We're just going in circles at this point. But I'll try one more thing. I'm going to quote from the actual standards that we are talking about.

In AMPQ there is a publisher/producer and there is a consumer. The words "Producer' and "Consumer" are right in the AMPQ standard:

"Producers and consumers are the elements within an application that generate and process messages."

The producer and the consumer do not need to be online synchronously. The broker is a means to an end. It is analogous to an HTTP proxy. Let me say that again: the broker is just a MEANS TO AN END, and the end is getting a packet from a producer to a consumer. The broker is also in the standard, but it is a means to an end for getting messages from producers to consumers ASYNCHRONOUSLY.

e.g. the standard mentions "Due to the potentially asynchronous nature of sessions, it is possible that both peers simultaneously decide to end a session."

In HTTP there is a client and a server. This is specified IN the standard. The standard also demands that they be online at the same time. Do you agree that the HTTP server must be available when the HTTP client contacts it?

I mean you can use Amazon's SQS through their HTTP API...by your logic that would make HTTP async.

Can you please show me where in RFC 9110 or RFC 2616 it mentions "Amazon SQS"?

If not, it's irrelevant. Because we're talking about the protocols defined in those standards documents.

I hope your patch gets accepted! That sounds like a big improvement!

You can love theoretically indigenous people without wanting them to have unique rights, but I admit that its a very tricky needle to thread. So is wanting them to have unique rights, however. There is no easy solution.

What makes a protocol synchronous in modern terminology (as opposed to older, hardware-oriented terminology) is whether the sender and receiver need to be online at the same time --synchronously. Whereas an asynchronous protocol like SMTP or Kafka works even if the sender and receiver are NOT online at the same time. Not synchronously online or available.

Protocols in the former camp benefit from simplicity, because the sender knows when the receiver has gotten the message, and can benefit from other data data in the reply.

Protocols in the latter camp benefit from reliability, because transient outages or slowdowns on either side do not cause lost data or cascading slowdowns. You also don't need to code retries into every app.

Yes, there are older definitions that talk about timing-oriented protocols. That's not what we're talking about in distributed enterprise systems.

https://www.akamai.com/glossary/what-is-an-advanced-message-queuing-protocol-amqp

"Another advantage of using AMQP is its ability to facilitate asynchronous communication among services. ... Generally, AMQP is most effective for sending asynchronous messages or notifications that don’t require immediate delivery or processing. "

https://learn.microsoft.com/en-us/dotnet/architecture/microservices/architect-microservice-container-applications/asynchronous-message-based-communication
"When using messaging, processes communicate by exchanging messages asynchronously. A client makes a command or a request to a service by sending it a message. If the service needs to reply, it sends a different message back to the client. Since it's a message-based communication, the client assumes that the reply won't be received immediately, and that there might be no response at all."