They aren't really. It's basically like getting an auto generated ref/branch name for each new commit, with convenience rewriting every ref when you rebase.
It's a bit more convenient if you prefer referring to a non-leaf commit directly rather than relative to the leaf branch a la master~2
change ids are more analogous to commit messages than branches, eg suppose that in a feature branch you have a "Delete deprecated classes" commit; in git there is a clear idea of "cloning" this commit (eg rebase, cherrypicks, maybe reverts) and the common sense that the new commit inherits the same commit message. Change ids are the same thing but in hex id form that can be created for every new commit/stash/index.
They allow for example to identify all the clones of a commit and they allow to give stable identities across rebases eg suppose you rebase a typo at the beginning of a feature branch without change ids a reviewer sees n new unrelated commits while with change ids it is possible to clearly identify which commits where changed/added/removed since the previous review iteration.
> suppose you rebase a typo at the beginning of a feature branch without change ids a reviewer sees n new unrelated commits while with change ids it is possible to clearly identify which commits where changed/added/removed since the previous review iteration.
A rebase can introduce change to a commit in cases such as handling conflicts or squashing.
Also, a commit already retains it's commit message after rebasing.
> A rebase can introduce change to a commit in cases such as handling conflicts or squashing.
and with change ids you can quickly separate commits that changed from commit that did not.
> Also, a commit already retains it's commit message after rebasing.
but commit message are not ids, there is no command for checking out a commit by its message, nor any sense that commit with the same message are somehow functionally related
Another thing that becomes easier with change IDs is reviewing multiple related commits together, essentially "stacked pull requests".
If you treat a branch as your unit of review, then it becomes super difficult for someone to submit a chain of related changes. You'll be constantly rebasing your pull requests onto each other as you get feedback from dependent branches.
I heard that the github CLI recently introduced support for this, but since in git there's no concept of dependent branches (a branch isn't even an object in git, just a reference to a commit), I think this approach will always be clunkier than reviewing commits related by a change ID.
> If you treat a branch as your unit of review, then it becomes super difficult for someone to submit a chain of related changes. You'll be constantly rebasing your pull requests onto each other as you get feedback from dependent branches.
From the paste, it's pretty obvious that they're using Linux with a custom llama.cpp fork. The Framework Desktop uses the AMD Strix Halo chipset, which is a unified memory system with an integrated GPU. I think that it goes for around $3000 in these inflated times.
In fact it says the opposite - that there is pruning.
Our brain also has some capacity limit, and maybe degraded memory performance over time, but in either case it's a graceful degradation - you may forget fine details of things that happened a long time ago etc, but you don't forget how to ride a bike just because it's been a while.
Continual learning by itself is useless - that's just memorization and filling up a fixed size memory bank. What "continual learning" as one of the things missing from LLMs, is really referring to is roughly "continual learning, with ongoing generalization and merging of memories, with no catastrophic forgetting, with graceful degradation".
This is slop. 8M parameter dense model with context length 64 that you train on enwik9 in 2h will have 1.15 bpb. This model has 1.8 (bits per byte, lower is better).
> Monarch Hadamard MLP: replaces the dense FFN with three learnable Walsh-Hadamard-initialized Kronecker (Monarch) factor pairs interleaved with per-channel diagonal scales, fixed permutations, a SiLU nonlinearity, and a rank-8 input-conditioned gate, so each token gets a fully mixed nonlinear transform of its d_model channels at O(d√d) parameters and compute instead of the O(d²) a dense 4x-expansion MLP would cost.
Wow, I was just researching W-H in transformers. Did yours seem to work? In my experiments swapping various components for W-H-like transforms caused extreme quality degradation.
UPD. according to the comments here, this model simply does not work at all, so I guess the answer is NO
reply