u/joinr

How did he slip up and get caught?

I can believe that - with enough constraints and tuning of an objective function - you can sufficiently constrain an optimization problem to find not only an optimal but a singular point in the solution space.

Given how often this last part (a singular solution) fails to emerge in actual high dimensional optimization problems, the complexity of the state space in programming, and the randomness serving as a mechanism to enable exploration, it sounds more plausible that you'll end up hopping between local optima; assuming a feasible solution emerges.

The randomness goes away entirely given you stay on top of what your agents are doing and constantly steer them towards your desired implementation

This reminds me of the old "With a sufficiently smart compiler....." line that used to get tossed around :)

I really enjoy optimization but not so much management. Maybe it's a useful mental reframe going forward.

It's not a deterministic system from everything I see. Fruit of the randomness tree.

But a stochastic system is predictive within a particular context, like a Markov chain that predicts the next most likely character given a particular set of characters.

This still sounds like a process driven by uncertainty due to randomness. Maybe there is some slippage due to the synonymous use (in many contexts) between stochastic and random, due to stochastic depending intimately on random.

But a stochastic system is predictive within a particular context, like a Markov chain that predicts the next most likely character given a particular set of characters.

IIRC MC is coupled with MC, wherein the second MC stands for Monte Carlo due to the random nature of the walk. Specifically, you are using transition probabilities coupled with a random walk to generate a sample path. I haven't done this stuff recently, but that's what I recall.

But a stochastic system is predictive within a particular context, like a Markov chain that predicts the next most likely character given a particular set of characters.

Sure. If you generate enough paths maybe you glean something from that and develop a predictive model about how other paths will likely unfold.

When I look at them, both require a source of randomness to model the uncertainty (via random variables) and in turn both are non-deterministic.

This is a fundamentally different process .... most similar to a genetic algorithm where you specify the constraints and the system evolves to meet them.

This sounds like (generally) local search, which also relies on randomness to explore new points in the solution space.

To me, your position is made clearer when you contrast a guided (stochastic) search for a solution vs. an unguided play of a game of chance. Both involve sources of randomness (e.g. both processes are random), but they differ in execution and intent (and hopefully outcome).

Also, it appears that you don't understand the difference between random and stochastic given that you keep using the term bingo.

Please explain how the three terms "random" "stochastic" and "bingo" relate to form this misunderstanding.

I use spacemacs. during setup, I choose the emacs mode (instead of the evil/vi bindings) and just go with other defaults. open a .clj file and it'll install the clojure layer (cider + more) and just works.

only other change I make is to turn on CUA key bindings instead of the legacy emacs ones, so your muscle memory for copy/paste/undo carries over. I think I had to make one other change (in the user init for spacemacs) to make sure C-z is undo, but I forget.

Pretty much sets itself up and it's consistent if I get on a new machine. CIDER is badass

I think borkdude is who agents send requirements to.

It sounds like most of your gains stemmed from fixating on the nix-style process oriented model with short-lived processes, vs. the jvm style long-running process model. I think you mostly just avoided repeatedly paying the typical clojure jvm startup tax (which is amortized for long running processes, or avoided with bb like you mentioned - at the cost of losing the jit and some language features [unless you're on nodebb which kind of snuck jit back in])).

An alternate processing model would have been 1 clojure process working on multiple files in parallel to saturate system cores. This is really trivial, but it's something you explicitly ignored because as you self-described "I'm lazy." AI could've also done it for you....so a bit odd to avoid using what's available. I think it's worth learning about since "an efficient and robust infrastructure for multithreaded programming" is one of the design goals of the language.

If you search "clojure parallel" and the immediate AI recommendations gets you there pretty fast. The naive versions are just parallel mapping, or parallel map/reduce, which is really common. So common (in fact) that you can do the same thing in most modern languages including Rust (which support task parallelism via libraries or core lang features). Clojure has a lot of additional support via libraries too.

The other potential benefit of a the single-process-but-multi-threaded model that saturates your resources is that you can exploit shared memory concurrency (where the multiprocess model has to work around process boundaries with side channels like mmap and some other tricks, although it's harder IMO on the jvm). I have no idea if your work benefits from this though (sounds like no, or at least your architecture precludes it).

I suspect leveraging the task parallelism that's already available (even built in naive-but-approachable stuff like clojure.core/pmap) probably would've let you stay in clojure and bypass the rust port (and inclusion of gnu parallel for task parallelism) while achieving the throughput gains. I am very surprised (given how much you leverage AI) that this wasn't a suggestion early on in your path.

You mentioned not knowing the difference between REPL and nREPL (when talking about jolt). As demonstrated in your book it's the foundational library (since early clojure) the establishes a network repl protocol for doing remote evaluation (waaaay before socket repl showed up IIRC), and most IDEs leverage it. So when you tell the reader to "jack in" either in Calva or whatever, it's typically an nREPL session that everything talks over (either locally or even on remote systems). You can see the connection initiated in the terminal output.

https://github.com/nrepl/nrepl

Regarding other comments here, I didn't have a problem following along with your vid (both sound and language).

Java class facade + ability to register shim "classes" such that jvm clojure interop code can be made to work without change (on top of your shims).

I think the scicloj community does this semi-regularly, or would be interested in results. Have you coordinated with them? Most discussion is on zulip threads https://scicloj.github.io/docs/community/chat/

Is this glojure or gloat doing the aot work [doesn't really matter]? will be cool to see the updated docs. runtimes look great and align with native comp.

Glojure doesn't fall back to an interpreter during AOT. If it can't emit native Go source, it generally fails compilation. During REPL development it relies on an AST tree-walking interpreter instead. Currently it's able to compile more code to native Go than let-go, but at the cost of more reflection or dynamic dispatch.

I didn't see anywhere this behavior is documented. I must have missed it. As I read it, the readme (and the self-comparison at the end) all seemed to point to an interpreter. Is there an example of a compiled clojure->go program somewhere?

Looks like the AOT-compilation is bypassing any (fallback) interpreted mode and emitting static Go that can be compiled to native. The other options are still interpreters, where the interpreter is compiled and bundled (I think). IIRC, let-go might eventually get a JIT in there to work on the bytecode interpreter.

I think in the AOT compiled native variant (e.g. via cljgo build ...), you're emitting go source directly, as in https://github.com/muthuishere/cljgo/blob/main/pkg/coreaot/core/core.go

So if you have closed-world assumptions, you can just compile to go instead of embedding a clojure interpreter, and one would hope, reap the performance and portability of go.

[edit]
I hadn't noticed the let-go is doing IR lowering for AOT when it goes for a native binary and emits go. So in that case, I guess the two are on a similar footing then.

maybe "collapsing towers of interpreters" adjacent

I was able to grab the windows release and compile a hello-world program that came in at about 9.1Mb, ~160ms execution time with direct-linking and tree-shaking and opt build (at least I used the options and noticed reduction in file size, so I think they're working). No other binaries beyond the joltc release. Similar program in bb is about 60ms. I am curious if it can be faster (I'm guessing since bb is interpreted, maybe it'll always be a little faster on startup).

Very cool.

using proxy is probably going to bottleneck performance at some point. It's meant more as a flexible wrapper to enable super class / inheritance stuff without need AOT and gen-class, but the implementation doesn't really care about performance.

Maybe something like https://github.com/redplanetlabs/proxy-plus would be better, or getting away from need proxy at all.

Or make your own class in java that does the extension by wrapping an instance of an interface (like IActor) with a ctor that takes the object-to-be-wrapped, then on the clojure side you can reify/defrecord/deftype as necessary with some convenience wrappers. I had to do something similar when wrapping an inheritance-based lib called Piccolo2D.

Based on the jira ticket, I think it's something like this:

(let [the-map  {:a 1 :b 2 :c 3 :d 4}
      {:keys [a b]
       :keys! [d]
       :select selected} the-map]
  [a b d selected])
;;[1 2 4 {:a 1 :b 2 :d 4}]

"Destructure the-map, lookup the keys for :a, :b, and :d, and bind them to lexical vars with the same name. key :d must exist. For all the keys that were bound during destructuring, dump them into a separate map and bind it to 'selected' "

instead of

(let [the-map  {:a 1 :b 2 :c 3 :d 4}
      {:keys [a b]
       :keys! [d]} the-map
      selected (select-keys the-map  [:a :b :d])]
  [a b d selected])

I think the win is that by replacing the explicit select-keys call, everything is coupled in one place inside the original destructuring form. So there's no ability to mismatch keys or make sure the select-keys form is kept up to date; it's taken care of at macroexpansion time for you.

It's meant to be a little fast-start static analyzer (e.g., a small binary that startup fast/on-demand) that only looks at clojure source to lint etc. Launching a thread in-process assumes you have a clojure jvm process connected, which means you paid the startup tax for Clojure. I think you can still do this with clj-kondo (e.g. have the linter running on demand on a jvm process), but it's not incorporating anything about the runtime environment in its analysis. Since its view of the world is static, it can quickly apply static rules to the source code and make suggestions etc, probably fast enough to keep up with realtime edits and stuff (I haven't used it, but I'd imagine the computational requirements are meager).

If you're writing independent tooling that supports automation (like deployments, verification, IDE suggestions etc), then having a little binary that can be invoked and completed before a clojure process even loads is useful. E.g., it can operate as a short-lived on-demand static analyzer (as opposed to other stuff like IDE tooling that analyzes in conjunction with like live nRepl information in a long-running interactive session).

There are some interesting libs in https://www.clojure-toolbox.com/ that can be educational on their own (just going through examples) under "Distributed Programming".

In general, an interesting resource is https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321

I use https://hazelcast.com/ via a modified https://github.com/tolitius/chazel wrapper for work stuff, but its mostly for interactively farming out work on clusters and not so much for the data grid stuff.

regarding your link, almost all of the databases will have either java or clojure wrappers or be approachable from FFI if they haven't already. Clojure has several of its own takes on databases (like datomic and xtdb). A lot of the streaming stuff is already wrapped too.

There's a recent effort by the guy that built the original Storm implementation (before the apache people rewrote 2.0 in java with an optional clojure API) to have a cloud-native db/processing/hosted thing called Rama https://redplanetlabs.com/docs/~/index.html#gsc.tab=0 . I haven't gotten into it, but the creator is a long time clojure person and is very enthusiastic.

You can get ~13x faster if you avoid the boxing in that (as well as just sticking with longs).
Can get to 23x if you avoid all the l2i casts, and just emit int-friendly array code and primitive stuff with bytecode emitter of choice (I used JiSE), as well as open possibilities for autovectorization by avoiding stock loop/recur and using native for loops (JIT only really looks there to see if there's a possible SIMD transform).

Would be nice to compare with a known vectorized solution though, to see if JVM is leaving anything on the table.

>everything that currently exists is remaining free, and only the new singleplayer content will have a price
tag

I feel like there's possibly a buried lede here that has shown up in other open source projects. "Everything that is 'free' today will remain 'free'," but going forward there appears to be an accompanying non-free presence with new terms of use and distribution. Specifically, this non-free relates to game assets (e.g. models, textures, missions, maps, and any various content that isn't scoped to the GPL stuff).

So, what about the future? Is this one of the paid off-ramps whereby the original free model is ship-of-Theseus'd over time while the official versions get EULA'd behind propietary assets? One can see a version where the once-free product is eventually obviated by the paid one (unless you have explicit requirements in place to provision licensing of any non-free assets for use in the companion free multiplayer project - I haven't really seen anything laying this out). Is there anything stopping Hooded Horse just replacing all the CC-BY-NC-ND licensed content (and any similarly authored work) content with new stuff owned by Hooded Horse. In that case, the source code is still available, but the meat of the game and its assets are now proprietary (absent a community alternative that maintains parity, e.g. if you add new units or features that are required for multiplayer that didn't exist prior to the split).

In concrete terms, does that mean that any new assets funded by the publisher (e.g. visual re-design or incidentally as part of the campaign effort, or anything else funded for purposes of profit), will be limited to the Premium steam version, or is there any plan to permissively share those assets with the Free Edition?

Conversely, if people want to contribute stuff explicitly for non-commercial work, is there a mechanism to do so without running afoul of the publisher's concerns?

I'm glad the BAR folks figured out how to make money off of this. Very cool project.

>vs tech.ml.dataset — TMD is more feature-rich (date handling, statistical functions, interop with many formats). Flatiron is smaller, has no native dependencies, and focuses on raw speed for a narrower set of operations.

What native deps are in TMD? I have not noticed any in my use (pure jvm environment due to security restrictions).

Someone recently said "AI is a force multiplier, but there has to be a force to multiply."

Yeah I looked at that initially. Downside is you wreck interop since there's an artificial conversion that happens, that the caller now has to respect. Best situation is just being able to pass strings around and have the clojure side guarantee its ops are immutable (although strings can still be mutated by CL stuff at the end of the day).

If you dgaf about interop, then yeah there could be marshalling to and from native to clojure strings.

I think it's unavoidable and just a continuation of the low-cod/no-code trend that has been pushed since forever. In that respect, the question is seemingly moot (e.g., it's not going anywhere, so it's on the honor system for people to disclose if they leveraged "assistance" for anything they submit going forward).
The tech is interesting and even appearing useful. If it enables you to achieve more than you could otherwise, then that's cool.

I don't like when people pretend to be great at spelling when they use spell checkers and autocomplete, or the analogous variation for passing llms and agents off as skill. Like the paradox of "I wrote this ancient egyptian algebra symbolic computation system in just 10 minutes" juxtaposed with "I haven't written a line of code in months, I just had the thing do it!" etc. Too much Lance Armstronging for me. That will also have a negative effect on creatives to share their stuff openly, so I'd anticipate a lot more closed source moating going forward, or free to use but closed. It'll probably lead us toward a parasitic model where actual innovation is hoarded and then mined for training data. Intellectual vampirism.

Then there's the whole unanswered angle of the implications of leveraging massive derivative work generators with (by default) 0 attribution. Given the legal system's legacy of embracing IP, I anticipate - eventually - substantial legal problems down the road despite the current rosy prospects of the token sellers "promising" to cover your legal costs and fight for you. I get the feeling when someone decides to rip off ("prompt a clean-room implementation") of some major IP, then we will see the big $$ fly and court cases mount. That might increase the pucker factor more. Hasn't happened yet though.

If someone wants to contribute to anything I have (not that I'd ever expect that hah), I think it'd be a hard requirement that they document "exactly" what portions of their product they (presuming they is an organic human) wrote vs. delegated to a derivative work generator. Just as with anything they wrote, they'd have to ensure whatever was generated complied with applicable licenses and assume personal liability for their contribution (indemnifying me etc.).