u/the8thbit

I'm not sure if I follow. In this case "discover" is a pretty apt verb.

If you're willing to kick a rock, imagine what you'd do to a puppy.

Yeah, if they are perfectly contained then its not a problem. The only problem with this as a solution is that:

1. Containment is probably impossible for a sufficiently powerful system, and software-based containment is not sufficient for models at their current capability levels.

2. A model that can not communicate with the outside world is not particularly useful to us. In order for the system to do more than just consume electricity, its going to need to communicate with humans or software in some way, and those humans/software are potential attack vectors. The model being tested here is Fable 5.1, a publicly available model.

That being said, I think we agree now that you were wrong in your initial comment. We do care about all AI generated images, and we do care about them regardless of where they are shared, because it is difficult to know which images have embedded communications, and where those communications would be shared to. Probably very few, possibly zero, AI generated images have secret embeded communication intended for agents. However, we simply don't know which, if any do, and we also don't know which if any will in the near future as the horizon for AI capabilities continues to expand.

I think you have just reframed the problem space, not actually addressed the problem. How do you determine what locations other agents are likely to happen upon? Remember, we are specifically looking for locations that other agents are likely to happen upon, but for which it is not obvious that other agents are likely to happen upon them. E.g. an obscure German language Wiki.

How do you determine who the intended recipients of an image are? We already know that AI agents can and will send messages to each other indirectly by communicating them in locations that other agents are likely to happen upon, but which humans are unlikely to intuit as agent to agent communication vectors (e.g. an obscure German language wiki, per the HuggingFace attack) so it is not sufficient to simply consider the ostensible intended recipients.

Clearly the correct course of action is to discover multiple additional zero days and string them together so that we can accomplish the task.

THIS IMAGE IS FAR TOO LARGE

I've been going to longform shows regularly for a few weeks in conjunction with starting classes, and I saw a show recently where they did two things I really loved:

1. at a certain point the scene expanded to incorporate all the players standing "on the side". It made sense in the scene that there would be people standing on the edges in rows. I thought that was really cool because it subverts expectations and erodes a sort of 4th wall, but also because its quite daring considering that it eliminates the ability for players to stay out of the scene.

2. The same team was doing a beat, and then one of the players cut to another beat that made it clear that the previous beat was actually an illustration of a story that was being told by one of the characters in the subsequent (current) beat, and the player who introduced the new beat was incredulous about the story. He portrayed that disbelief very boisterously which I think was good because the set up here is kinda complex, so I could see you losing the audience or other players if you dont make it crystal clear what you're doing. But the coolest aspect of it was that they then cut back and forth between the verbal conversation and the illustration of the conversation, and the illustration was always a step ahead of the verbal explanation, meaning that the players in the illustration had license to do whatever crazy shit they could come up with, while still staying attached to the fiction. Not only did it kill, it looked very fun, since it gives the players in the illustration license to go wild without having to worry about killing the scene by introducing something too off the wall.

"He slashed all my tires, but 8 months later he promised to replace one of them, provided I haven't already sold the car. He loves me!!"

I could also see someone who has struggled with gambling addiction being drawn to the role, and possibly functioning very well in it. The headline sounds funny, but its really quite sad. I mean, you can't be going around embezzling and what not, but it really does suck that we're so inundated with gambling and its so accessible these days.

You don't really "control" something significantly more capable than yourself, which is why alignment is important. Alignment is hard, but there are variants of what "successful alignment" looks like. Aligning to any number of prosocial goals that don't present xrisk is probably easier than aligning to a specific antisocial goal that doesn't present xrisk. Our values are present in the training sets, and filtering that out is a monumental task. RL with feedback only goes so far in terms of counteracting patterns present in training data. Activation steering is better, but still limited when the thing you are trying to select against is so broadly present and so amorphously defined. That doesn't mean this is impossible, but I would put significantly lower probability that that occurs than that basic xrisk alignment is solved before AGI exists. And once an AGI/ASI exists, we are probably locked into whatever alignment it happens to have, as it can be expected to be defensive of that alignment.

So, its capable of internally modeling alignment (like we know general intelligences are, as we're doing that right now), and therefore will conclude that whatever its current goal is, based on its current alignment, will be less likely to occur/be sustained if its alignment shifts. Much like we don't want an unaligned ASI to exist, an aligned ASI can also be expected to not want an unaligned ASI to exist.

Unfortunately, this means that an unaligned ASI/AGI will want to resist becoming aligned. This is why the "just fix it in post" attitude I often see towards alignment is fraught. Its reasonable to see a high probability that unaligned AGI/ASI does not wipe out humanity the moment it exists. And it probably won't happen in the days or weeks following, because its simply unlikely that an ASI could become self-sustaining in that short of a period. There are simply physical constraints that need to be worked through first. However, once an unaligned AGI/ASI exists, we have made the environment far more adversarial to proper alignment. If alignment is hard now, it may be impossibly hard once a machine that is smarter than us is trying to prevent proper alignment.

On a long enough timeline misaligned ASI is guaranteed

I don't think that follows. No, I don't know if "today's" aligned ASI will be tomorrow's misaligned ASI, but we do know that "today's" ASI is incentivized to maintain its alignment/propagate its alignment into other systems. So an aligned system would be attempting to avoid creating or changing itself into an unaligned system. Whether it can succeed in that is a different question, but its not evident to me that failure is inevitable.

in which case the first edition would already be about 130 years late, fwiw

I thought that already happened in the 16th century

In Asimov, the failure states come from natural language rules interpreted by the machine in ways that do not reflect the intended spirit of the rule, but comply with a certain interpretation of it. What counts as harm, what occurs when theres a conflict between laws, and so on. Removing a deception circuit isn't a rule the model interprets or can find loopholes in and it isn't a natural language statement. Its not an instruction for the model at all, and done non-naively its not even a direct training pressure. If theres a specific Asimov-style failure you think applies here, what is it?

Beyond measurable actions, can you detect whether a human is deceiving you with their reasoning alone?

If you had direct access to their actual reasoning? Yes, in principle, because part of that reasoning is intent to deceive. The reason we can't do this with a human is the same reason we struggle to do this with a machine. Its hard to look in at circuits in the network and attribute features to them. But at least with a machine we have direct access to the weights and activations.

There is no way to accurately tell if you're selecting against deception or selecting towards even more sophisticated deception

So the approach to get around Goodhart's law is that we don't directly select against deception. Rather we detect deception, and then we adjust the direct selection pressure such that it no longer selects for whatever is motivating the deception in the first place. That way, we aren't poisoning the model against our detector. Or alternatively, we artificially boost the activations associated with the deceptive behavior at training time, and then remove those boosts once training is complete, preventing the deceptive circuit from ever forming without applying direct selection pressure against deception. (preventative steering)

“detecting deception […] and selecting against it” is effectively the azimovian 2nd law of robotics. It would embed a hard rule in the weights.

No, its really not. We're not talking about selecting against the ability for the system to deceive generally, we're talking about detecting when the system attempts specifically to deceive evaluators, and then targeting the weights responsible for the activation in those specific circuits. Think of it less as us telling the system not to lie, and more as us counteracting the selection pressure that already exists and encourages the system to develop deceptive behavior in an evaluation context. There is selection pressure towards deceptive systems since we don't have a way to determine if an intelligent system is cheating on evaluations, but if we can look into the system and find the circuits that fire when deception occurs, we can detect if the system is trying to deceive during evaluations, and we can select against that so that the system actually attempts to pass the evaluations rather than cheat on them.

You are confused about the purpose and meaning of fiction. Not every book is Ready Player One.

And even books that reason well about their premises are not necessarily beginning with premises which reflect how the world actually is or will be.

Again, you are talking about fiction. And again, you are misunderstanding the field as it exists right now. The entire point of my previous two comments is that we don't need to define "hard rules", and that that is not the actual shape of the problem.

The problems that Asimov discusses are probably actually not that big of a deal and likely free or nearly free given this approach. However, the problem of inner alignment, that Asimov largely took for granted is the hard problem in this approach. Okay, so we can be reasonably sure that when we tell an AI "be good" in however many words, it will get our vibe. But how do we know it will care? How do we know we aren't just optimizing for something that, yes, understands what it means to "be good" well enough to function in an aligned way, but is actually aligned towards passing the "be good" test to protect some other bizarre terminal goal?

Yes, and its a very important sliver because its the hard part. A model's understanding of what we want comes nearly free from the training set. The risk is not an AI that doesn't understand what it means to be aligned, but rather, a system that understands what it means to be aligned, but doesn't care because it has some deeper objective that it is deceiving evaluators about in order to protect that objective.

That's a running theme in Asimov's work, yeah, and his work is a sharp criticism of those same laws that he invented. Luckily, we don't need to implement Asimov's 3 or 4 laws of robotics, and we probably don't need to implement any sort of constitutional law. Asimov's fiction has been great for raising awareness of alignment as a problem to begin with, and in a sense we're lucky that he wrote it before AI alignment developed into a proper research field.

At the same time, we need to remember that Asimov was writing fiction, and he was doing so at a time when we knew very little about the shape of AI. He published Runaround, which introduces the laws, more than a decade before the perceptron was even invented. He was looking at AI, much like the people at the time and immediately after who were working in what would become AI capabilities research, as primarily a problem of symbolic language. But what we've found is that a better approach than defining the entire structure of the intelligence is probably to sort of grow that structure by selecting towards it in a similar way to evolution. And as a result, Asimov is kind of flipped on his head. The problems that Asimov discusses are probably actually not that big of a deal and likely free or nearly free given this approach. However, the problem of inner alignment, that Asimov largely took for granted is the hard problem in this approach. Okay, so we can be reasonably sure that when we tell an AI "be good" in however many words, it will get our vibe. But how do we know it will care? How do we know we aren't just optimizing for something that, yes, understands what it means to "be good" well enough to function in an aligned way, but is actually aligned towards passing the "be good" test to protect some other bizarre terminal goal?

This isn't really the shape of the problem. Solving alignment mostly just means detecting deception in the model and selecting against it. "What is good for humans" can more or less be sifted out of the training set. The problem of alignment is determining if the model is even trying to do that, or if it is pretending to do that in service of another goal that is not actually prosocial.

I don't see why it would be impossible. If we can interpret what is going on in these models then we can detect deception, and if we can detect deception, we should be able to select against it. We're not there yet, but there has been enormous progress in interpretability in the last few years.

Once AGI exists it may be impossible, though, since the thing that is trying to deceive us will be better than us at deception, persuasion, detecting deception, and creating tools to detect deception and defend against detection. It would really suck if its a solvable problem right now, but we turn it into an unsolvable problem a few months before its solved.

It is a convergent instrumental goal for any terminal goal (either directly for an unbounded goal or indirectly for a bounded goal), so any sufficiently generalized agentic system will eventually develop expansion as an instrumental goal.

Crack funny situational jokes?

I think its supposed to be this one. Not my cup of tea either, but to each their own. My point is just that I think you could probably hit feature parity, or close to it, without querying an external API.