I come with a background in computer architecture and embedded Linux. I was interested in ML systems and was reading up about it, and after going through several job postings and this link , noted that optimizing compiler code is a requirement. My questions are -

1) Is knowledge of how compiler front end is written and IR code generated required to optimize them?
https://engineering.purdue.edu/online/courses/tagged_items?q=compiler

2) What's a good resource for compiler optimizations that can help in ML systems?

https://www.cs.toronto.edu/~pekhimenko/courses/cscd70-w18/docs/Lecture%201%20[Intro]%2001.11.2018.pdf

3) Are learning compiler optimization techniques the same for LLVM and MLIR? I don't see a lot of resources for MLIR compiler optimization. Is learning optimizations on LLVM helpful for MLIR and is learning LLVM not so useful for ML code optimizations ?

Thank you.

I've been working on a compiler-like system for a while now, and it's gone through many stages of evolution.
A consistent pressure early on was away from multiple representations: I started with many switch statements and many kinds of representation for each stage, and ended up with one kind of node and handler based dispatch per stage.
Yet, when I look at (most) other compilers, their construction is far more static in nature, and far more varied in terms of the kinds of things presented. I've found forms like SSA impede my ability to reason about optimizations rather than aide them, and I've had enormous success from simply retaining and annotating one structure rather than continually converting.
A specific case, to provide one, is liveness propagation. Because values are already shared between their occurances (Nodes) and already have a system for acquiring properties (Quals), I can simply mint liveness tokens onto the children of expressions with output, and it automatically propagates.
Though I've not focused too much on codegen and optimization, my main goal with the compiler is extensibility and syntax flexibility.
So why not retain the AST? Turn it into a structure worth annotating and preserving from which optimizations and codegen can be performed more easily?

I wanted to get the opinions of others on this matter, I'm open to challenge.

Some more explanation can be found here: https://goldensystems.ca/GDSL_core

I’ve been building a small Lisp compiler written in C++ and compiles S-expressions directly to native x86-64 assembly, using a tiny runtime for things like printing integers, doubles, and strings.
Currently it has functions, arithmetic, integers, doubles, strings, etc.
The compiler is still pretty simple:

S-expressions → AST → semantic analysis → x86-64

No VM, no bytecode — just Lisp turning into machine code.
I’m also starting to look into adding a small IR between the AST and codegen as the language grows.
Mostly doing this as a learning project and because writing a Lisp compiler seemed like a fun rabbit hole. :)
Github: https://github.com/xms0g/tinysexp

I reached the point in my compiler that I can add C++ Interop, the AI has a long detailed plan. LLVM+ClangAST will do the parsing to types that is mapped to my language syntax.

Reason for C++ support is open existing library support written for C++, not all of them have C API. A search for existing languages rejected or abandoned C++ Interop the high cost to maintain and the language stability.

Anyone else have experience?

LLevo un mes picado contra malbolge jajajaja y creo he logrado algunas cosas jajaja si quieren checar mi traductor, tengo el quijote completo en .mal, acepto todo tipo de critca construcitiva :P https://github.com/DannyBaanks/Malbolge-Translator

I'm working on the resource-control layer for a language that compiles effects (network, files later databases and concurrency primitives) down to thin near-zero-cost kernels instead of using an interpreter or a heavy runtime.

Current state: the compiled binaries basically just call the underlying syscalls. There's no pooling, no admission control and no unified accounting yet.

Two main approaches are being considered:

Per-effect arbitration. Every effect operation does a request/grant with a central (or sharded) resource manager before continuing.

Boundary-leased admission. Acquire a lease once at a boundary (accepted connection, opened file, spawned task entry, etc.) then let the individual operations on that resource run with a cheap local check.

The second approach keeps the path (send/recv/read/write) extremely light: local atomic check + direct kernel call + non-blocking telemetry emission. Telemetry is fail-open.

I'm especially interested in trade-offs

How coarse the admission boundary should be when you don't yet have a request-oriented server surface

Whether putting even a very cheap lease check on the hottest I/O path is acceptable

How this interacts with structured concurrency and deadlock detection (resource-acquisition graph vs reply-obligation graph)

Whether NFRs (latency, concurrency limits, error-rate targets) should be expressible, in the language itself and enforced by the same kernel

What have people found works (or fails) when adding resource control to low-overhead compiled effect systems? Any designs you'd strongly recommend or avoid?

I am currently working on Shaft, a new programming language. It is in it's bootstrap phase (C++) and build on LLVM. It already works, but still has lot's of issues and limited features. The goal is to make it safe and still readable; it also already has automatic memory cleanup in reverse declaration order, so you can move one field of a struct while the rest is still valid, and the compiler will generate the cleanup code, without the need of user-defined destructors. I’m looking for interested developers who want to help build a compiler and contribute features or bug fixes. example snippet:
```Shaft def add(i32 x, i32 y) ?-> i32 result { tunnel x + y -> i32 result; }

def main(String[] args) { reserve ?i32 result = add(4, 9); valid result { printf("4 + 9 = {i32}", result); } else { println("Addition failed"); } } ```

I made a small and simple post fix interpreted programming language called rdn, it's familiar to forth developers and developers who use Lua as a scripting language for their systems, rdn merge both of them, you can use it for writing scripts or for configurations or even query language

It's written in C and it provides a simple and friendly API for the developers

I would be happy to have you participate in this project

This is the GitHub repo:

https://github.com/abdorayden/rdn

Thank you

In numerical simulations and scientific code, developers often face a frustrating trade-off: write clean, expressive physics equations that run sluggishly, or write convoluted, unrolled, hand-optimized loops that run fast but become impossible to read and maintain. I built Strilight to bridge this gap. It doesn't pretend to introduce magic—it’s fundamentally a developer quality-of-life tool. You write your physical or mathematical concept in whatever natural syntax you prefer, and Strilight inspects the AST behind the scenes to solve the underlying recurrence relations in closed form: * $O(N) \to O(1)$ for scalar linear reductions, periodic shifts, and telescoping series. * $O(N) \to O(\log N)$ for multi-variable coupled recurrence systems via binary matrix exponentiation.

In Python (Just a single decorator):

python from strilight import accelerate @accelerate def compute_simulation(steps: int) -> int: acc = 0 for i in range(steps): acc += (i * 3) + 7 return acc In C (Via Developer Contracts & Pragmas): c long long compute_reduction(void) { long long total = 0; #pragma strilight accelerate target(total) include("config.h") for (unsigned long long i = 0; i < N_STEPS; i++) { total += STEP_INC; } return total; } How does it work on physical kinematics? When a particle or celestial body travels along an unperturbed trajectory (free flight, gravitational orbit, or steady acceleration), Strilight collapses the entire iterative time-stepping sequence into minimal algebraic evaluations—without sacrificing coordinate precision. When discrete collisions or boundary interactions occur, execution transitions into specialized coupling matrices.

Zero Risk & Decisive Fallback: Non-invasive: It's just a decorator or pragma. You can add or remove it at any time without altering your algorithm. Decisive Safe Fallback: If a loop contains unstructured side-effects, unknown external calls, or non-affine dynamics, Strilight decisively halts acceleration attempts and runs the native loop. It will never break or crash your program.

repositery link

Hello!

For past month about, I've been working on a stack based compiler (inspired by the Forth programming language). I have a few example programs (the larger ones like the raytracer and neural network being generated by Claude because I'm just not very good at thinking stack based, sorry for the slop) which show some of the features of the language.

Its architecture is a compiler/VM structure, sort of like how Java works. It has word definitions, control statements, loops, and more.

If you want to check it out, here's the link: https://github.com/SlothScript/stakku

I was answering a post about another language which had inflicted a lot of verbosity upon itself by requiring every variable be referenced as Variable[name] to distinguish from keywords.
This brought me to a thought I wanted to share, about a potential way to deal with this.
Before the token stream has been assembled into the AST, you could run a pass over it where each operator token checks it's adjacent neighbours and if they resolved as keywords that shouldn't normally resolve to values, turn them back into plain identifiers.
Now this is just a very basic concept I wanted to float here, to get some push back, and edge cases, because I find the concept interesting.

Hi folks, this is No Saved DATA. I dedicate this post to describe some of the features I put in Neve to make it an expressive high-level language (close to Python/PyTorch syntax), while also allowing efficient low-level code. I am sharing this now, because I believe the language has already strongs traits that allow it to be extended to other problem domains.

Current results:

• Close to Python/SentencePiece in text processing + Byte-Pair Encoding (BPE) training;
• Competitive with NumPy and OpenBLAS in CPU matrix multiplicaton, but with pure high-level SIMD code;
• It was able to train a CIFAR Resnet faster than PyTorch, but I did not debug whether this was due to better CPU or GPU orchestration. But the LSTM was slower (mine lacked kernel fusion and other optimizations). Also, that old Neve deep learning framework was mostly implemented in C++. I am now changing it to be mostly implemented in Neve. That is, compute intensive preprocessing, automatic differentiation, parallel dataworkers and GPU kernels all in high-level.

Besides, I recently added GPU Kernels code interface. However, the complete framework will still take some more months.

The current state is an evolution of a post I made some months ago in other subreddit (https://www.reddit.com/r/ProgrammingLanguages/comments/1ql585o/brand_new_nsk_programming_language_python_syntax/)

Links

📚 Documentation
💻 GitHub

────────────────────────────────────────

Intro

I started creating Neve after seeing the code of the Efficient Zero reinforcement learning model. It has a parallelism that PyTorch does not handle, and the implementation required using Cython packages for having threads (literaly coding in C, then just calling C functions from Python). Later, I realized PyTorch also needed to implement its data worker threads in C, another workaround over Python Global Interpreter Lock (GIL). Not only that, even preprocessing implementations like the BPE are made in C, C++, Rust, etc...

So, currently, people must choose between languages like Python for high-level productivity, C and relatives for compute efficiency, Lua for advanced interoperability and other languages for concurrency. Thus, since in my job I had to wait hours for my neural networks to train, I decided to create a programming language in the remaining time. One language that had all these features, which are of high value for deep learning research. Nowadays, I believe it matured to such a point that it may be extended to other complex problem domains.

Since Python syntax is very simple and has most of the users, I chose it as the basis. But it run a LLVM JIT in its background. Now I will explain important expressions and features in Neve.

────────────────────────────────────────

Finish/Async and Data Split

I experimented Jax deep learning framework for a while. During this period, I learned an expression that would take a tensor or a vector as inputs. It could vectorized the function over the first dimension. A threaded adaptation I made for Neve is:

def int foo(array<int> v)
    print("Thread ", tid, " has vector:")
    v.print()

main
    array<int> u = arange_int(2,20)
    finish
        asyncs 3 foo(>u)

This splits a vector across three threads, so it can be processed in parallel. This is useful when you have a list of files, and want a function to process the files across N threads.

────────────────────────────────────────

Channels

I saw fireship videos a long time ago about Elixir and Erlang. These languages have actor-message passing, which were used in scaling applications to massive concurrency. Then, this year, my advisor suggested me to study Go and Rust, so I could see the tendencies about modern languages. I got surprised by Go channels expressions, which I thought to be an evolution of the actor-message model (but in the end they solve different problems). Go also applies channels to green-threads (concurrency within a single OS thread), but I was happy with using it for standard threads.

Once I finally adapted Go channels to Neve, I was able to reduce some five lines of code in data loaders. Even if it was only five lines less, it got much cleaner.

  def float worker()
      print("Start worker")
      int yield_ptr, bs=self.batch_size
      print("worker ", tid)

      while self.load_ch.alive()
          yield_ptr = self.increment_yield_ptr()

          for b=0, b<bs
              self.getitem_w(yield_ptr+b, b)
              self.load_ch <- tid

          self.x.switch()
          self.y.switch()

  def tuple<gpu_tensor,gpu_tensor> batch()
      int w <- self.load_ch

      var x = self.x.load(w)
      var y = self.y.load(w)

      x = x.view([$cfg.bs, 1, 28,28])
      return x, y

These are functions from the dataloader class. The channel communicates which threads have data ready to be consumed. Then, the cpu tensors (self.x and self.y) can process and yield data using ping-pong buffers. It is much lower level than PyTorch, but without the need of implementing the underlying parallelism in C++. That gets rid of boilerplate mutexes and more than 100 lines of C++ code. Posteriorly, once Neve gets inheritance and interfaces, most of the parallel logic may be hidden, so it can be even closer to PyTorch.

The training code is already similar to PyTorch

        ...
      gpu_tensor a, b
      a, b = ds.batch()
      var y_hat = model.forward(a)
      ce_loss(y_hat, b)

      $backprop.backward()

────────────────────────────────────────

Anonymous Functions

This expression is crucial for mapping tensor operations to their respective backward ops.

def int add(int x, int y)
    return x+y

def int mult(int x, int y)
    return x*y

main
    map<str, Function<int, int, int>> m
    m["add"] = add
    m["mult"] = mult
    print(m["mult"](3,4))

────────────────────────────────────────

Generics

def T bar<T, U>(T x, U y)
    print("bar x: ", x)
    print("bar y: ", y)
    return x

main
    int z = bar(3,4)
    z = bar(5,"$%*OU")
    print("z ", z)

Generics may yield complex code, but may also save hundreds of lines when the same matrix multiplication function should be implemented for different data types (int4, int8, float16, bf16, etc...) (I still didn't test the generics in this scenario :p).

────────────────────────────────────────

Operation Overload

Defining new operations for data types is simple.

def gpu_tensor @(gpu_tensor a, gpu_tensor b)
...

Which works thanks to generics. The operation is consumed as:

var z = x @ y

For gpu_tensor types.

────────────────────────────────────────

Globals

Neve has no primary data type globals. Instead, global values can only be defined as unique instances of classes.

class Backprop
    array<BackNode> ops
    def float register(gpu_tensor l, gpu_tensor r, gpu_tensor out, str op)
        self.ops.append(new BackNode(l, r, out, op))

This defines the global Backprop class that holds the backs (backward function definitions). Then, any tensor operation may use the global instance of Backprop to keep track of the operations to execute later.

def gpu_tensor @(gpu_tensor a, gpu_tensor b)
    ...
    $Backprop.register(a, b, ret, "mma")

Once Neve finds an "$", it automatically inserts in the main an instruction to create a new instance of that class, so it can be used everywhere. Althought standard global values are not supported, this expression forces global variables to belong to a common scope. It helps preventing pollution/confusion versus standard global vars. For example, you could put all your globals inside a class named Config, then use any of its values.

$Config.ip

It is straightforward to spot it belongs to a global scope.

────────────────────────────────────────

GPU Kernels

import nsk_cuda

gpu void @(
        layout<bf16, m, n> x, layout<bf16, p, n> y,
        float[] z
        )
    ...

kernel void mma_kernel(bf16[] x, bf16[] y, float[] z, int M, int N, int P)
    var v = layout<bf16, M, N>(x)
    var u = layout<bf16, P, N>(y)

    z += v[256,N](bx,0) @ u[128,N](by,0)

This one tiles z, v and u, storing the matrix multiplication result in the tiled z positions. The operator overload recovers a function that has shared memory async copies, which are overlapped with tensor core operations, all described in Neve itself.

The layout expression is subject to change, but it won't be too much different from the current.

────────────────────────────────────────

Interoperability and Libraries Support

In the early stage I was very inexperient with programming languages, so I tried to implement all my important functions and composite data types in C++, and call the functions from Neve. The negative side was that the quick sort was orders of magnitude slower than Python. The positive, I made a C++ tokenizer and parser to extract LLVM bindings.

NSK had a heavy focus in using C++ bindings for functionalities. Now it is almost unnecessary, as basically everything can be designed in Neve itself.

Use C++ interop when you:

- Need system calls only found in C++ (you may create a library that maps these calls to Neve);

- Want a custom memory allocator (I used this one for GPU mallocs/memory arena).

The way Neve adopts C++ functions:

extern "C" int float_cpu_print(Scope_Struct *scope_struct, void *tensor, DT_array *vec) {

After compiling and importing, the functions map naturally to Neve functions and data types. For example, the expression:

x.print()

Will call any function named float_cpu_print, given that x is a float_cpu. That implementation could either be defined in C++ or Neve.

Functions that have composite data types require explicit prototypes in Neve, in order to extract the nested type. But if a function takes a composite data type as argument, it is better to define it in Neve when possible.

It also allows adding LLVM extension functions in C++, which enable using LLVM for generating IR directly. Besides, it is possible to add new LLVM data-types.

C++ and LLVM functions must be compiled to dynamic libraries, and their make require linking system packages. The documentation has a in-depth guide on how to make them work, and the youtube channel has some tutorials about it as well.

Overall, I recommend building libraries in Neve itself. You can import libraries using imports in the current directory.

import my_nv_file
import my_lib/my_nv_file

These import other .nv files. It is also possible to turn them into packages if you organize them under ~/.local/neve/lib/<my_pkg_name>, then import as:

import my_pkg_name

If you get into the my_pkg_name folder, you can commit it to github, then anyone can install it with

nsm install <my_git_user>/<my_pkg_name>

Nsm is automatically installed along with neve when executing the bash install. It works for both Neve and C++ compiled packages (more testing is necessary).

────────────────────────────────────────

Other Features

• JIT: it feels like Python to execute code - no need for compiling files. Meanwhile, it has the JIT speed benefit;
• Packet manager;
• Concurrent garbage collector;
• Syntax highlight for vim and vscode;
• Very simple/incomplete LSP, tested in neovim only.

────────────────────────────────────────

Limitations

• There are still very rare crashes in large codebases, like in the BPE after executing it many times (due to that stupid concurrent garbage collector);
• Works in Linux only, because I couldn't get LLVM to work in Windows;
• Still lacks inheritance and interfaces.

────────────────────────────────────────

I hope you enjoyed the tour. Ready to test?

wget -qO- https://github.com/NoSavedDATA/Neve/releases/download/neve-bin/install.sh | bash

I have been building this entirely solo so far. Let me know what you think of the syntax choices, especially the approach to parallelism and GPU kernels!

Do you think Neve can help you in your domains?

A little while ago I posted about the v0.1 release of ABC, a small compiler and programming language I originally developed for teaching.

I shared it here, on Hacker News, in a few other communities, and also in a German-speaking subreddit. The discussions made me think about a question I had not really considered when I started the project:

What should happen to ABC now?

For its original purpose, the project is essentially a success. It has done what I wanted it to do in my teaching, and actually exceeded my expectations.

One concern that came up in the German discussion was roughly:

Nobody raised that point here, but I suspect some people may have had the same thought. :-)

I've been using ABC for two years now in HPC0, my undergraduate Introduction to High Performance Computing course. HPC0 is an elective. In the following winter semester I teach HPC1, which is mandatory in some programs and elective in others.

HPC1 uses C++ throughout. We do things like cache-optimized matrix multiplication, LU factorization, multithreading, MPI, CUDA, etc.

My observation so far is that students who took HPC0 have a noticeably easier time in HPC1. Some of them had hardly programmed at all before HPC0.

Of course, that's not scientific evidence. There is an obvious selection bias: HPC0 is elective, so the students taking it may simply be more motivated to begin with.

But my underlying argument is that there are a number of fundamental concepts you need to understand really well. Once those concepts are in place, transferring them to C++, Rust, or another language is comparatively easy.

My deliberately provocative version is:

Either you can program or you can't. Once you really can, the particular programming language becomes mostly a tool.

The interesting educational question for me is therefore: How do you get someone to the point where they really can program?

That's what ABC is for. It was never meant to be the language students would use for the rest of their professional lives.

And since I'm already being provocative: sometimes I get the impression that the generation that learned programming with Pascal was the last one that was actually taught how to program. :-D

I'm very happy to be challenged on that one. ;-)

So I now see two possible futures for ABC.

The first is straightforward: declare the experiment essentially finished.

I could extend the C ABI support a little further, implement it for ARM64 as well, improve a few things, and leave the project as a reasonably complete teaching compiler. The raylib examples already demonstrate that the language and compiler can be used for more than tiny classroom examples.

That would be a perfectly satisfactory outcome.

But there is another possibility that I find much more interesting:

Could we build a small modern language that plays something like the role Pascal once played?

A language designed to teach programming in a way that leaves you not merely knowing a language, but understanding concepts that transfer to other languages and remain useful throughout your career.

I think those skills may actually become more important rather than less important in an age of AI-generated code. Even if someone eventually does a lot of “vibe coding”, somebody still needs to understand what the machine is doing, why something is slow, why memory gets corrupted, or why generated code doesn't behave as expected.

But I don't want a language that is useful only for teaching.

I'd like it to be possible to write genuinely useful programs with it.

The ideal is still what the original name suggested: “A Better C.”

Small enough that you can understand the language and its implementation, close enough to the machine that you can explain what happens, but without preserving every historical accident of C.

There are a few language features I have been considering:

• Compile-time evaluation / something along the lines of constexpr, plus inline functions. This would eliminate many of the common reasons for C preprocessor macros: constants, small max-like functions, etc.
• Modules.
• Inline assembly. I needed this when experimenting with ABC on bare metal on an ATmega328P. For example, consider implementing a delay as something conceptually as simple as:

​

fn delay(n: u16)
{
    while (n--) {}
}

Now things suddenly become interesting. n needs to be handled appropriately in registers, and the compiler must not optimize away a loop that has no observable effect according to the normal language semantics.

I like examples like this because they force you to understand the boundary between language, compiler and machine.

There are probably a few more language features I would add.

But deliberately not many.

The goal would not be to slowly turn ABC into C++.

A language that leaves the classroom would also need tooling.

A formatter analogous to clang-format would be useful, as would proper LSP support.

There is already some preliminary work in this direction. Last year I supervised a bachelor's thesis in which a resilient parser was developed. It can't simply be dropped into the existing compiler, but there is at least a prototype of one important component that can be used for experiments.

And my experience from developing ABC so far is that some of these things become usable surprisingly quickly if you start small.

But there is one thing I don't think I can do alone:

Turn it from my project into a community project.

I can continue developing ABC as the language I use in my courses. But if it is supposed to have a life outside my classroom, I don't think it should simply remain “Michael Lehn's language”.

It would need people who experiment with it, criticize it, discuss language design, build tools, write examples, and eventually make decisions I would never have thought of myself.

So this post is partly an experiment:

Do you think there is room for such a language?

Would any of you be interested in participating in its design or implementation — even just through discussions and experiments at first?

And perhaps there is an amusingly concrete first community problem we could solve:

The language needs a name. :-D

“ABC” (A Better C) worked fine for a university teaching project, but the name is obviously already taken. If the language is going to leave the classroom, that starts to matter.

My current brilliant idea is “emsiel”, a phonetic rendering of MCL — Michael C. Lehn.

There is just one minor flaw with that idea: if the goal is to turn this into a community project, naming the language after myself might not be the most promising first step. :-D

So perhaps that's actually a good place to start:

What would you call a language like this?

Compiler/project: https://github.com/michael-lehn/abc-llvm

I am working on my own compiled programming language, and I figured that a decent test suite would save many headaches down the line (especially as my current code is in desperate need of refactoring). The problem is, I am not sure how to test a compiler without just mindlessly writing a bunch of end-to-end tests as that would be slow and unlikely to catch obscure bugs

While implementing the OO part of my language (AET), I ran into a performance problem: OO method calls have overhead. So I started looking into devirtualization.

At first, I treated it as a compiler problem: how can the compiler determine that a method call has only one possible target?

But then I started thinking from a different angle: what if the language itself could tell the compiler that the target is unique?

This made me realize that the relationship between language semantics and compiler shouldn't be one-directional. They should influence each other during the design phase:

Language Semantics ↔ Compiler ↔ Optimization

For example, AET has:

private$ foo();
final$ foo();
final$ class A { ... };

These are language semantics that restrict inheritance and overriding. But they also provide the compiler with clear semantic guarantees: the call target is unique.

A final$ method cannot be overridden by subclasses.

A final$ class has no subclasses that could override the method.

A private$ method does not participate in overriding at all.

Different language rules, but from the compiler's perspective, they all provide the same useful fact: the call target is unique. So AET can use this semantic information to transform an OO call into a direct call to the corresponding FUNCTION_DECL in GCC's intermediate representation.

Of course, a compiler could also discover the same information through type analysis, call graph analysis, devirtualization, LTO, etc. But if these facts can be determined directly by language semantics, could it in turn make the compiler simpler?

This led me to a more general question. Essentially, it's a "who does more, who does less" problem. If language semantics provide more explicit guarantees, the compiler may need to do less inference. If the language keeps weaker semantic constraints, more work falls on compiler analysis.

So the question becomes: what should be left to language semantics, and what should be left to compiler analysis? Are there any methods or theories to guide this division of labor, to make it more scientific and reasonable?

I think this is also a boundary worth discussing between language design and compiler design. AET is my exploration of this question while actually implementing it.

Would love to hear your thoughts.