u/snaphat

We absolutely do not understand LLMs to that degree. Not to be pointed but you're conflating knowing how the underlying computations work with understanding the internal mechanisms and emergent behaviors that result from them.

There's an entire field of research (Intrinsic Interpretability) dedicated to figuring that out, with a substantial body of published research on the subject.

E.g. https://arxiv.org/abs/2504.08001

We do have established methods for measuring divergent thinking, though that doesn't mean they're without limitations:

https://doi.org/10.1037/aca0000691

LLMs have demonstrated divergent thinking in controlled experiments, though that isn't the same as generating novel knowledge.

Also, having a PhD isn't a source. That's why I avoid bringing up my credentials unless they directly inform the reasoning behind my argument.

I wish folks on Reddit would stop relying on appeals to authority, particularly when it comes to LLMs. Credentials alone are rarely sufficient evidence, and often aren't even relevant to the specific claims being made

Consciousness itself is poorly understood, and we don't even have an agreed-upon definition or criteria for determining what possesses it. How can we conclusively determine what lacks consciousness when we haven't established which properties are necessary or sufficient for it?

Knowing how an LLM is implemented doesn't resolve that question. We understand its architecture and underlying computations, but that doesn't mean we fully understand its internal representations or emergent behavior, much less whether any of those processes could give rise to subjective experience.

The difference isn't simply learning from examples. Humans use inductive, deductive, and abductive reasoning to develop working models of how things operate, derive conclusions, and infer explanations for what we observe. We also have metacognitive abilities that allow us to critically evaluate our own reasoning, question our assumptions, recognize contradictions, and revise our understanding when something doesn't add up.

LLMs can perform all three forms of reasoning to varying degrees, but they're particularly unreliable at complex, multi-step deduction and abductive reasoning involving competing explanations or underlying causal relationships. They can develop internal representations, but that doesn't necessarily mean they have a coherent or consistent model of what they're reasoning about.

More importantly, their metacognitive capabilities are limited. They often struggle to independently evaluate whether their own reasoning is correct, recognize flawed assumptions, or identify contradictions in their conclusions.

These are fairly concrete limitations that help explain why we wouldn't necessarily consider them hyper-intelligent in the same sense as someone like Einstein, even when they can perform tasks in seconds that would take a human hours.  Being able to process enormous amounts of information or produce results quickly isn't the same thing as having robust, generalizable reasoning and the ability to critically evaluate your own understanding.

This gif is the one good meme of 6 7

You are wrong again, midi was largely used in windows 95 and 98 with the general midi protocol made by Microsoft, and by 16bit games , same thing for 3dfx with most games being windows only instead of dos

What a strange aside... midis and 3dfx are not related to DLL injection. I never mentioned midis or 3dfx in any form at any point.

Your response does not logically connect to what I said. It kind of seems like word salad or madlibs. It makes me wonder if you are a bot

so you should stop about this useless dos argument.you should stop selling those compatibility layers, those are off topic in a decomp reddit.

? Come on... lol

Anyway, original topic mentioned dos games indeed, and indeed dll injection is impossible in them like it was already mentioned in previous topics so why continue?

You specifically brought up DLL injection into 16-bit applications. You were either referring to 16-bit Windows applications, or you didn't understand that DOS applications don't natively support DLLs.

The funny part is that DOS could also run 32-bit applications using DOS extenders, so if you were actually talking about DOS, why specify 16-bit DLL injection? Their lack of native DLL injection support has nothing to do with bitness, and plenty of games on the platform were 32-bit

Context is key.

If I were a college student in an unrelated field (say, accounting), then my opinion should hold less weight on the matter.

Mmm, simply stating you have 30+ years of experience as a developer or software engineer doesn't qualify you to make broad, sweeping claims about LLM capabilities.

The problem is that while your 30+ years of experience in software development may establish your general expertise in the field, the vast majority of that experience predates modern LLMs.

When it comes to observing and evaluating LLM capabilities, you've had essentially the same timeframe to gain experience as any other software developer who has been using them since they became widely available in the last three or so years. Your 30-year career doesn't automatically give your observations about LLM performance any particular authority over those of other developers with comparable experience using them.

It would be kind of like a car mechanic claiming that their 30+ years of experience gives them particular authority to evaluate electric vehicles, even though they've only been working with them for the same amount of time as everyone else. Their experience with conventional vehicles is relevant to their general mechanical expertise, but it doesn't automatically translate into greater expertise or knowledge of a newer technology, nor does it necessarily give them any particular insight into broader trends involving that technology.

These studies are interesting, but I literally work in this space every single day. A study that claims LLMs currently underperform the average programmer on code review is simply wrong; it would be like if we had a study that said cars can't travel faster than 80km/h. I literally drive faster than that on a daily basis.

Personal experience, or argument by anecdote, is not sufficient grounds to dismiss systematic empirical evidence. Personal incredulity is not a valid reason to dismiss years of empirical research simply because its findings don't align with your own observations.

There's a reason anecdotal evidence is generally considered one of the weakest forms of evidence: it's susceptible to selection bias, confirmation bias, and uncontrolled confounding variables.

How do we know whether or not those biases and confounding variables influenced your observations? Well, we know they did. They always do in uncontrolled environments. The question is only by how much. Nobody can reliably determine that from your anecdotes, and there's no particular reason for anyone to trust that your observations weren't irredeemably biased. Having 30+ years of experience certainly doesn't establish that they weren't.

Moreover, the very fact that you have such strong opinions about the technology raises additional concerns about confirmation bias, particularly when you're outright dismissing empirical evidence simply because it contradicts your personal observations.

Not trying to be pointed here, but that's just the nature of why anecdotes are inherently unreliable as evidence, and why we use controlled studies, blinding where feasible, and other methodological safeguards to minimize the influence of these biases and confounding variables.

Separately, your car analogy fundamentally misrepresents what the studies are claiming. A study finding that LLMs underperform human reviewers on average isn't claiming that LLMs are incapable of outperforming individual reviewers. Observing exceptions doesn't contradict an average-performance comparison.

It would be like rejecting a study finding that one category of cars is slower on average than another simply because you've personally driven cars from the supposedly slower category faster than some cars from the other.

Furthermore, even supposing one's personal observations are entirely accurate, they don't necessarily generalize to a broader population. No one has a reliable way of establishing that their observations do generalize without systematic empirical evidence. Feelings are not evidence.

I think there's an even broader point about observation that needs to be made here. You have no particular reason to trust your own conclusions any more than anyone else does, given that you have no reliable way of knowing how much your own biases have influenced them. Humans tend to give greater credence to their own observations simply because they're their own, not because they're necessarily more accurate, reliable, or objectively justified. That is the fundamental weakness of beliefs based on personal anecdotes, and why people should strive for better, more reliable forms of evidence wherever possible to mitigate the natural, inherent biases in their own cognitive processes.

If you believe the studies are wrong, the appropriate response is to identify flaws in their methodology, measurements, or applicability. Simply declaring them wrong because your personal experience tells you otherwise isn't a meaningful rebuttal

// stepping off the podium lol

OTVDM is maintained as well. And now you're talking about DOS games rather than 16-bit Windows applications, where DLL injection would apply.

All of this is neither here nor there. My original point was simply that OTVDM lets you run Win16 applications directly within modern Windows and that DLL injection into an already-running application might be technically feasible. I only went into more detail because you seemed to misunderstand how OTVDM works and how it's conceptually similar to WoW64.

If you were talking about DOS applications all along, code injection is possible there too, using real-mode techniques. But I reasonably assumed you meant Win16 because you specifically said, "yea try dll injection on a 16bit app and come back..." DOS doesn't natively support DLLs, so that wouldn't make much sense in the context of DOS applications

Yup, I'm aware DOSBox can run Windows 95/98, particularly DOSBox-X. That doesn't contradict anything I said.

The distinction is that DOSBox runs an entire guest operating system inside an emulated PC, whereas OTVDM runs Win16 applications directly within the host Windows environment, translating their API calls to native Windows APIs. It doesn't require installing or booting another copy of Windows or DOS

They're fundamentally different approaches, both in terms of integration and performance. Running DOSBox to emulate an entire PC, boot DOS, and then run Windows 95/98 on top of that just to execute Win16 applications introduces substantially more overhead than OTVDM's approach. lol

? That's not Wine. That's OTVDM (also called WineVDM). It uses Wine-derived components, but it's a compatibility layer for running 16-bit Windows applications on modern Windows, conceptually similar to WoW64 (Windows-on-Windows 64-bit), a compatibility layer Windows has used to run 32-bit applications on 64-bit systems for roughly 25 years.

Older 32-bit versions of Windows used NTVDM and the original Windows-on-Windows (WoW) subsystem for 16-bit compatibility. NTVDM could use the processor's Virtual 8086 mode (VM86) for real-mode DOS programs, while WoW supported 16-bit Windows applications running in protected mode.

Microsoft dropped these subsystems from 64-bit Windows, partly because x64 long mode no longer supports VM86 (in hardware). Technically, Microsoft could have retained Win16 compatibility through a WoW subsystem using the 16-bit protected-mode execution capabilities that modern x86-64 processors still support, but doing so would have required additional operating-system infrastructure.

OTVDM restores Win16 functionality through CPU emulation and API translation. It takes advantage of Microsoft's surviving NTVDM64 executable-substitution mechanism, allowing Windows to launch supported 16-bit executables directly without manually starting an emulator or installing an older copy of Windows. So, from a usability perspective, it's just about as native as you can get.

Ironically, modern processors are so much faster than the hardware these 16-bit programs were originally designed for (typically running at tens of megahertz at most) that they can run substantially faster even through software emulation than they did on native hardware of the time.

Anyway, it's intended for Windows applications, not DOS applications, and doesn't run natively on Linux. It isn't abandoned either. The project received a merge just 2 days ago: https://github.com/otya128/winevdm/commits/master/

EDIT: It's also worth noting that a key distinction is that OTVDM uses Wine-derived components to integrate legacy applications with the actual Windows APIs, rather than relying on Wine's reimplementation of those APIs (which is kind of why Wine historically has had so many compatibility issues with applications).

Instead of emulating an entire Windows environment, OTVDM emulates the necessary 16-bit CPU execution environment while translating Win16 API calls into calls to the host Windows operating system.

Not to contradict this but for folks that don't know, you can run 16-bit Windows apps on modern windows: https://github.com/otya128/winevdm

You can't use the standard DLL injection techniques that work with modern 32-bit and 64-bit applications. However, WineVDM appears to implement some of the mechanisms that Windows 3.1 used to load 16-bit DLLs and execute their code in another application's context, including cross-task Windows hooks. So it may be possible to inject a 16-bit DLL into an already-running Win16 application (I haven't tested whether this actually works)

Mmm... I don't think there's sufficient empirical evidence to justify the claim that LLMs are better code reviewers than all but the very best humans. There's plenty of both recent and older research documenting the limitations of LLMs, including those that persist in current models.

For example, LLMs can misidentify correct code as defective, and their judgments can be influenced by irrelevant changes to the code:

https://link.springer.com/article/10.1007/s10515-026-00638-5
https://aclanthology.org/2026.acl-long.888/

AI code reviewers also produce false positives and struggle with bugs that require broader codebase context:

https://arxiv.org/abs/2603.11078
https://www.microsoft.com/en-us/research/publication/when-should-a-software-factory-be-lights-off-benchmarking-code-review-agents-on-bug-detection/

Research also documents limitations in vulnerability detection and repair, even with newer models:

https://link.springer.com/article/10.1007/s10664-026-10812-8

And real-world evaluations have found high rejection rates for AI review suggestions, while faster AI-assisted reviews haven't necessarily resulted in better review quality:

https://arxiv.org/abs/2607.03316
https://arxiv.org/abs/2607.13196

Note that all of these studies were published in 2026, since some zealous proponents of AI tend to dismiss older research as outdated. There are plenty of studies from previous years documenting similar limitations, too. These aren't isolated findings; researchers have and continue to identify many of the same problems over time.

We'll see what the landscape looks like in another six months, but my guess is that researchers will continue to identify many of the same limitations. Eventually, we'll probably see a systematic review that examines which limitations persist across model generations and how their severity has changed over time. I seriously doubt that a systematic review conducted within the next five years will find that these limitations have been comprehensively resolved across the range of tasks involved in code review.

(As a side note, invoking your credentials here is weird because they don't provide any systematic evidence of how LLMs compare to human reviewers. It just comes across as a strange aside.)

Yeah, unfortunately you really do need someone familiar with the codebase to evaluate their findings. 

They often misunderstand complex bugs, particularly when the behavior depends on interactions across multiple systems. They can be very good at identifying a plethora of potential issues, but their proposed fixes tend to be lackluster. I often find that they suggest inappropriate, kludgy patches that address the immediate symptoms while missing the deeper root causes or broader architectural issues at play. 

Welcome to a world where emotional intelligence isn't taught in society, at the home, or in schools where we let our feelings override our better judgments lol

I think the response is deeply rooted in our brains though. There a bunch of theories on what could contribute to the desire:

https://en.wikipedia.org/wiki/Imaginary_audience https://en.wikipedia.org/wiki/Face_(sociological_concept) https://en.wikipedia.org/wiki/Reactance_(psychology) https://en.wikipedia.org/wiki/Impression_management https://en.wikipedia.org/wiki/Closure_(psychology) https://en.wikipedia.org/wiki/Social_comparison_theory

That's true. And it also devalues legitimately good uses of LLMs to the point that some folks can't tell the difference between slop and non-slop

If it were better and more consistent at software development, I don't think there would be such a public backlash in terms of the software side of things (except from the skilled programmers whose job is on the line)

I personally hope it improves (someday) to the point that the lack of expertise associated with vibers becomes less of a problem... I'd rather we just have higher quality projects regardless of whether a human wrote them or not instead of having to perform my own due diligence on every new project I see that i might be interested in.

Although theoretical improvements that make vibe slop indistinguishable from well designed and tested software  might not be a good thing in other ways foreseen or unforeseen. If LLMs were legitimately competent, without hand holding, it would probably be affecting employment and society much more directly than it currently is

Meaningfully extending software using injection and/or binary patching is hard and often requires extensive rewrites to the base functionality. So it can effectively end up being rewriting code to replace existing functionality in much the same way that a recomp would

It also doesn't solve cross platform support

Bro shut up and get to work double clicking those DOS executables, they aren't going display "this app can't run on your PC" errors by themselves 

Unfortunately, LLMs are often repetitive and disorganized in what they do output so longer language output will often have multiple copies of the same dense information in varying forms. It can be particularly bad for readmes.

It's the same kind of issue they have with duplication across codebases, where they'll duplicate superficially different multiline code across a codebase instead of maintaining a single copy 

It doesn't necessarily contradict the findings. It could be optimizing for information density in what it does output, but also meandering at the same time

The authors did not measure or control for topical drift, repetitive information, or unnecessary elaboration so I would take any broad conclusions with a grain of salt 

I mean, that'd be the best move even if they didn't block you, right?

But, it is these cases that make it understandable why folks want to reply even so

By low-level technical knowledge, I mean the implementation-level understanding someone would ordinarily need to perform the programming tasks they're delegating to an LLM.

You don't necessarily need to memorize the syntax of an API request, but understanding what an API is doesn't necessarily mean you know enough about a particular API to make informed architectural decisions involving it. Implementation details and technical constraints can directly influence those decisions.

Moreover, much public LLM use boils down to cases that are far more egregious than this where the user doesn't have a basic understanding of core 101 level programming concepts. So, we are seeing more and more pieces of software where the folks in charge are incapable of recognizing, evaluating, or fixing the issues, let alone something more complicated like attempting architectural design

I mean, sure, that's fine but much of the time that isn't really what is happening on reddit

It is often not the case that you have someone decide they don't want to converse with you and then just block you. That would be akin to an analogy of walking away... If it really was that way I don't think many folks would have a problem with it

Instead what you tend to get is someone who freaks out publically at you in a very antagonistic and rude way and then blocks you directly after

The more apt analogy to this case in RL would be someone freaking out at you in person and then "walking away" after having the public fit

That doesn't really happen in RL though. Having a public meltdown over a conversation with a stranger is generally not seen as appropriate in society and would cost you socially if not get you banned from public establishments for poor behavior 

Eh, in many cases making informed architectural decisions wrt programming requires low-level technical knowledge

Much of the time what precedes a block is someone throwing a tantrum in a comment right before they block someone so folks naturally want a chance to respond to the tantrum.

Like in RL if a random grown adult had a meltdown at you, you'd probably want the chance to at least respond... even if it's going to be fruitless