Repository navigation
simd/archsimd: architecture-specific SIMD intrinsics under a GOEXPERIMENT #73787
Description
Update (06/23/2026): Go 1.27 RC1 is released, which includes the experimental support of the portable size-agnostic simd package (#78902), an update of the archisimd API for AMD64, and the inclusion of archsimd support for Wasm and ARM64, all under GOEXPERIMENT=simd. There are some breaking API changes for AMD64 archisimd API; please update the code accordingly. See #73787 (comment) .
Update (02/11/2026): The AMD64 low-level SIMD package is now available in Go 1.26 under GOEXPERIMENT=simd. There were minor bug fixes since the RC2. We'll continue the development for Go 1.27 on the dev.simd branch. See #73787 (comment) .
Update (01/15/2026): The AMD64 low-level SIMD package is now available in Go 1.26 RC2 under GOEXPERIMENT=simd. The RC2 includes a few bug fixes, some of which involve API changes from the RC1. See #73787 (comment) .
Update (12/16/2025): The AMD64 low-level SIMD package is now available in Go 1.26 RC1 under GOEXPERIMENT=simd. Also, the package is renamed to simd/archsimd, per #76473,. See #73787 (comment) .
Update (08/20/2025): A preliminary implementation of AMD64 low-level SIMD package is being developed on the dev.simd branch. You're welcome to check it out and try it with your use cases. Feedback welcome! See #73787 (comment) .
Proposal Details
SIMD is crucial for achieving high performance in many modern workloads. While Go currently allows access to SIMD via hand-written assembly, this approach has significant drawbacks: it is difficult to write, prevents asynchronous preemption, and hinders inlining for small kernels.
Adding SIMD support in Go (without requiring writing assembly code) has been requested, see e.g. #35307, #53171, #64634, #67520. Here we propose a SIMD API and intrinsics without a language change. It has some similarities with #53171, #64634, and #67520, with differences in the details.
Two-level approach
Generally, Go APIs are often simple and portable. However, the SIMD operations that the hardware supports are inherently nonportable and complex. Different CPU architectures support different vector sizes and operations, and sometimes have different representations.
A portable SIMD API would be nice. As mentioned on #67520, the work on Highway (a portable SIMD implementation for C++) demonstrated that it is possible to achieve a portable and performant SIMD implementation. For C++, it is built on top of architecture-specific compiler intrinsics, which we don't yet have for Go.
Our plan is to take a two-level approach: Low-level architecture-specific API and intrinsics, and a high-level portable vector API. The low-level intrinsics will closely resemble the machine instructions (most intrinsics will compile to a single instruction), and will serve as building blocks for the high-level API.
It is expected that most data-processing code can just use the high-level portable API and achieve good performance. When some uncommon architecture-specific operations are needed, the low-level API is there for power users to use and perform such operations.
Another way to look at them, as mentioned in #67520 (comment), is that the low-level API is analogous to the syscall package, whereas the high-level one is analogous to the os package. Most code that interacts with the system will use the os package, which works on nearly all platforms, and occasionally some code will reach into the syscall package for some uncommon system-specific operations.
In this proposal, we focus on the low-level architecture-specific fixed-size vectors for now, using AMD64 as a concrete example. Variable-size vectors (scalable vectors, e.g. ARM64 SVE) support and a portable high-level API will be addressed later. We propose adding this under GOEXPERIMENT=simd for a preview.
Design goals
Here are some design goals for the low-level architecture-specific API.
- Expressive: Being an architecture-specific API, the lower-level package can cover most of the common and useful operations that the hardware supports.
- Despite being a low-level API and intended for power users, we expect it to be relatively easy to use. It is expected that general users could read and understand code using this API without digging into the hardware details.
- Best-effort portability: When an operation is supported on multiple platforms, we intend to have a portable API for it. However, strict or maximal portability is not the goal for this level. In most cases, we don't plan to emulate operations that are not supported by the hardware.
- It will be a building block for the high-level portable API.
Portable(-ish) vector types with possibly architecture-specific methods
The SIMD vector types will be defined as opaque structs. The internal representation contains the elements of the proper type and count (we may include other zero-width unexported tagging fields if necessary).
package simd
type Uint32x4 struct { a0, a1, a2, a3 uint32 }
type Uint32x8 struct { a0, a1, a2, a3, a4, a5, a6, a7 uint32 }
type Float64x4 struct { a0, a1, a2, a3 float64 }
type Float64x8 struct { a0, a1, a2, a3, a4, a5, a6, a7 float64 }
etc.
The vector types will be defined on the architectures that support them. The compiler will recognize them as special types, using the vector registers to represent and pass them.
We do not define them as arrays, as often the hardware does not support element access with a dynamic index.
Operations will be defined as methods on the vector types, e.g.
// Add each element of two vectors.
//
// Equivalent to x86 instruction VPADDD.
func (Uint32x4) Add(Uint32x4) Uint32x4
which performs an element-wise add operation. The compiler will recognize it as intrinsic, and compile it to the corresponding machine instruction. E.g. on AMD64, it will be compiled to the VPADDD instruction.
The operations will be defined in an architecture-specific way (controlled by build tags). This allows different architectures to have different operations. Common operations (like Add) will be defined on almost all architectures.
Naming
We choose the names of the operations to be easy to understand, and not tied to a specific architecture. The goal is that general readers will understand the code without digging into the CPU details.
Common operations (like Add) will have the same names and signatures across architectures. If an operation is not supported by the hardware on some architectures, it will not be defined there. We will avoid methods with the same name and signature but different semantics on different architectures.
The machine instruction name (like VPADDD) will be included in the comment, for the ease of lookup.
Load and store
The machine instructions will load/store from/to a pointer. For type safety it should be a pointer to a properly sized array type.
func LoadUint32x4(*[4]uint32) Uint32x4
func (Uint32x4) Store(*[4]uint32)
Loading/storing from/to a slice are expected to be common useful operations, which will be defined from the above, e.g.
func LoadUint32x4FromSlice(s []uint32) Uint32x4 {
return LoadFloat64x4((*[4]uint32)(s))
}
(LoadUint32x4FromSlice is long. Maybe we name it LoadUint32x4, and name the array pointer form LoadUint32x4Ptr?)
Another option is for the load operation to take a placeholder receiver, like
func (Uint32x4) Load(*[4]uint32) Uint32x4
func (Uint32x4) LoadFromSlice([]uint32) Uint32x4
The receiver's value will not be used, just for providing the type. This form keeps the name short. But this form is confusing, in that x.Load may appear as a load to x (updating x) except it does not (it returns the loaded value as a result), and is not idiomatic as a Go API.
Mask types
As pointed out in #67520, the internal representation of masks is different from architecture to architecture, and even different at different levels of CPU features. E.g. on AVX512, a mask is one bit per element of the vector and stored in a mask register (K register); on AVX2 it is stored in a regular vector register with one element per vector element; and on ARM64 SVE, the mask is one bit per byte of the vector.
To handle this situation, we represent masks as opaque types. The compiler will choose the representation that is appropriate for the program. A mask can be used in an operation that supports masking out some elements, and in logical operations between masks. It can also be explicitly converted to a vector. E.g.
func (Uint32x4) AddMasked(Uint32x4, Mask32x4) Uint32x4 // VPADDD.Z
func (Uint32x4) Equal(Uint32x4) Mask32x4 // VCMPEQD or VPCMPD
func (Mask32x4) And(Mask32x4) Mask32x4 // VPANDD or KANDB
func (Mask32x4) AsVector() Int32x4 // VPMOVM2D or no-op
Besides operations that produce a mask (like comparison), masks can also be created from a vector, or from a bit pattern (on AMD64, it will be AVX512 style, 1 bit per element; on ARM64 SVE, it will be a different function getting a mask from a 1-bit-per-byte pattern.)
func (Uint32x4) AsMask() Mask32x4 // no-op or VPMOVD2M
func Mask32x4FromBits(uint8)
The compiler will choose the representation depending on how the mask is consumed, and will remove conversions between different representations if possible. E.g. for an Equal operation followed by an AsVector, the compiler will choose VCMPEQD which produces the result directly in a vector register. For an Equal operation followed by AddMasked, it will choose VPCMPD, which produces the result in a K register, which can be consumed in a VPADDD instruction.
It is not recommended to use the masks directly in other ways.
Alternatively, we could have separate, explicit types and methods for different representations of the masks, e.g.
func (Uint32x4) EqualToVec(Uint32x4) Uint32x4 // VCMPEQD
func (Uint32x4) EqualToMask(Uint32x4) Mask32x4 // VPCMPD
This way, the methods are more directly corresponding to machine instructions.
Note that the opaque mask representation doesn't preclude the addition of the more explicit representation and methods. If later it turns out that the explicit methods are needed, we can add them under different names.
Some masked operations (e.g. in AVX512) support two modes of masking: zero mask, where the unselected elements are set to 0, and merging mask, where the unselected elements remain unchanged in the destination register. The merging mask is more complicated, in that the destination register is actually both an input and an output. Zero mask has a simpler API. Therefore we choose the zero mask version (thus VPADDD.Z above). To use the merging mask operation, one can use the zero mask version of the operation followed by a blend operation with the same mask, and the compiler can optimize to a merging mask operation. For example, x.AddMasked(y, m).Blend(z, m) can be optimized to a merging masked instruction VPADDD x, y, m, z.
Conversions
Vector elements can be extended, truncated, or converted between integer and floating point types. E.g.
func (Uint32x4) TruncateToUint16() Uint16x8 // VPMOVDW
func (Uint32x4) ExtendToUint64() Uint64x4 // VPMOVZXDQ
func (Uint32x4) ConvertToFloat32() Float32x4 // VCVTUDQ2PS
In some cases it may be useful to truncate to fewer or expand to more elements of the same type, or to "reinterpret" a vector as another vector with the same bits but different element type and arrangement. Some such conversions don't need to generate a machine instruction.
func (Uint32x8) AsUint32x4() Uint32x4 // truncate
func (Uint32x4) AsUint32x8() Uint32x8 // expand, zero high elements
func (Uint32x4) AsInt32x4() Int32x4 // unsigned to signed
func (Uint32x4) AsFloat32x4() Float32x4 // interpret the bits as float32
Constant operands
Some machine instructions require a constant operand, or have a form with a constant operand. For example, on AMD64, getting or setting a specific element of a vector (VPEXTRD/VPINSRD instruction) requires a constant index. For shifts, one form is shifting all the elements by the same amount, which has to be a constant (the other form being shifting a variable amount for each element). Namely,
func (Uint32x4) GetElem(int) uint32 // VPEXTRD
func (Uint32x4) SetElem(int, uint32) uint32 // VPINSRD
func (Uint32x4) ShiftLeftConst(uint8) Uint32x4 // VPSLLD
It is recommended (and will be documented) that these methods are called with constant arguments, so it can generate efficient code. What happens if it is not? The corresponding C intrinsics may cause a compilation failure. We could choose to do an emulation, or generate a table switch (as the range of the constant operand is usually small), as a fallback path.
An (incomplete) API list
To give a concrete example, here is the proposed API for Uint32x4 on AMD64.
type Uint32x4 struct { a0, a1, a2, a3 uint32 }
func LoadUint32x4(*[4]uint32) Uint32x4 // VMOVDQU
func (Uint32x4) Store(*[4]uint32) // VMOVDQU
func (Uint32x4) Add(Uint32x4) Uint32x4 // VPADDD
func (Uint32x4) Sub(Uint32x4) Uint32x4 // VPSUBD
func (Uint32x4) Mul(Uint32x4) Uint32x4 // VPMULLD
func (Uint32x4) Min(Uint32x4) Uint32x4 // VPMINUD
func (Uint32x4) Max(Uint32x4) Uint32x4 // VPMAXUD
func (Uint32x4) And(Uint32x4) Uint32x4 // VPAND
func (Uint32x4) Or(Uint32x4) Uint32x4 // VPOR
func (Uint32x4) Xor(Uint32x4) Uint32x4 // VPXOR
func (Uint32x4) AndNot(Uint32x4) Uint32x4 // VPANDN
func (Uint32x4) ShiftLeft(Uint32x4) Uint32x4 // VPSLLVD
func (Uint32x4) ShiftLeftConst(uint8) Uint32x4 // VPSLLD
func (Uint32x4) ShiftRight(Uint32x4) Uint32x4 // VPSRLVD
func (Uint32x4) ShiftRightConst(uint8) Uint32x4 // VPSRLD
func (Uint32x4) RotateLeft(Uint32x4) Uint32x4 // VPROLVD
func (Uint32x4) RotateLeftConst(uint8) Uint32x4 // VPROLD
func (Uint32x4) RotateRight(Uint32x4) Uint32x4 // VPRORVD
func (Uint32x4) RotateRightConst(uint8) Uint32x4 // VPRORD
func (Uint32x4) PairwiseAdd(Uint32x4) Uint32x4 // VPHADDD
func (Uint32x4) MulEvenWiden(Uint32x4) Uint64x2 // VPMULUDQ
func (Uint32x4) OnesCount() Uint32x4 // VPOPCNTD
func (Uint32x4) LeadingZeros() Uint32x4 // VPLZCNTD
func (Uint32x4) Equal(Uint32x4) Uint32x4 // VCMPEQD or VPCMPD $0
func (Uint32x4) GreaterThan(Uint32x4) Uint32x4 // VCMPGTD or VPCMPD $6
func (Uint32x4) Blend(Uint32x4, Mask32x4) Uint32x4 // VPBLENDMD
func (Uint32x4) Compress(Mask32x4) Uint32x4 // VPCOMPRESSD
func (Uint32x4) Expand(Mask32x4) Uint32x4 // VPEXPANDD
func (Uint32x4) Permute(Uint32x4) Uint32x4 // VPERMD
func (Uint32x4) Broadcast() Uint32x4 // VPBROADCASTD
func (Uint32x4) GetElem(int) uint32 // VPEXTRD
func (Uint32x4) SetElem(int, uint32) uint32 // VPINSRD
func (Uint32x4) TruncateToUint16() Uint16x8 // VPMOVDW
func (Uint32x4) TruncateToUint8() Uint8x16 // VPMOVDB
func (Uint32x4) ExtendToUint64() Uint64x4 // VPMOVZXDQ
func (Uint32x4) ConvertToFloat32() Float32x4 // VCVTUDQ2PS
func (Uint32x4) ConvertToFloat64() Float64x4 // VCVTUDQ2PD
func (Uint32x4) AsInt32x4() Int32x4 // no-op
func (Uint32x4) AsFloat32x4() Float32x4 // no-op
For the operations that support masking, we will also have masked versions, like AddMasked above.
Some operations are supported only on some types. For example, a vector with floating point elements (e.g. Float64x4) supports Div, Reciprocal, and Sqrt, but does not support shifts or OnesCount.
Note that this list is not comprehensive, e.g. it doesn't yet include operations like Gather, Scatter, Intersect, or Galois Field Affine Transformation. We do plan to support these operations in the API. This proposal is just proposing the direction we're heading to. We can add more operations later.
Note also that we intentionally don't want to include some forms of the machine instructions in the API, and instead leave them to the optimizer. For example, on AMD64, many arithmetic operations support a memory operand. Just like the language has a + operator and a dereference (unary *) operator but not an operator for a + *b, we don't provide an API for the same operation on vectors. Instead, if a Load operation is followed by an Add, the compiler can optimize it to the memory form of the ADD instruction.
CPU features
SIMD operations often require certain CPU features. One may want to check if specific CPU features are available on the target hardware, and guard the SIMD code with it. CPU feature check functions will be provided, e.g.
func HasAVX512() bool
func HasAVX512VL() bool
The compiler will treat them like pure functions, as they never change after runtime initialization. That said, it is still recommended to check CPU features before doing SIMD operations, instead of doing it in the middle of a sequence of SIMD operations.
It is an open question whether we want to enforce that a CPU feature check must be performed before using a vector intrinsic, through static or dynamic analysis. Required checks would encourage portable code (across machines of the same architecture with different CPU features).
AVX vs. SSE
On AMD64, there are two "flavors" of SIMD instructions, SSE and AVX. Almost all the SSE operations have equivalent AVX instructions. It is not recommended to mix the two forms, which may sometimes result in a performance penalty. Therefore, we want to stick to one form. The initial version of this API will always generate the instructions in AVX form. One goal of this API is to support writing performant code on the evolving hardware, making use of advanced CPU features. SSE operations may be added later in an explicit or transparent way, if there is a strong need.
Discussion
Alternatives
We have considered a few alternatives APIs.
Instead of methods, the operations could be defined as top-level functions, like AddUint32x4. This name is unnecessarily long, and repetitive if a piece of code operates on the same type of vectors over and over. We also considered defining them as generic functions, like Add[T Vec](T, T) T. This makes the name short, but in user code it is likely that the package prefix (simd.) will still be repeated. The main difficulty for generic functions is that it is hard to express relationships between types, e.g. MulEvenWiden returns a vector with half the number of elements with doubled width. Besides, the implementation will be more complex. Overall it doesn't provide much benefit over methods.
We also considered defining the vector types as generic types, like Vec4[T]. This approach has similar difficulties as generic functions. It is also hard to express irregularity. For example, on AMD64, SaturatedAdd operation is supported with 8- and 16-bit integer elements, but not wider ones.
Future work
Scalable vectors and high-level portable API
A number of architectures have chosen to adopt scalable vectors, such as ARM64 SVE and RISC-V Vector Extension. For scalable vectors, the size cannot be determined at compile time, and it theoretically can be quite large. We plan to add support for scalable vectors in Go, although currently we're not ready to propose a concrete design.
On top of that, we plan to add a high-level portable API for vector operations. Existing portable SIMD implementations such as Highway will be a source of inspiration. To support various architectures and CPU features, the API will probably be based on scalable vectors. On platforms like AMD64, it may be lowered to a fixed size vector representation, depending on the hardware features.
It is expected that for the majority of use cases in data processing and AI infrastructure, it will be possible to write just with the high-level API, which achieves portability and performance. We also hope that the low-level and high-level APIs are interoperable. If some code is mostly portable but just needs an operation or two that is very architecture-specific, one can write the code mostly using the high-level API, and drop to the low-level just for these operations.
Activity
dominikh commentedon May 19, 2025
How do you plan to handle immediates, like in
VPINSRD? Specifically the requirement that their values be known at compile time?randall77 commentedon May 19, 2025
These should generate
Uint16x4andUint8x4? Otherwise these would be more of aReinterpretoperation?Not a fan of the "pairwise" name, or the weird semantics (receiver goes to low 2 entries, arg goes to high 2 entries). Maybe
Where to get a
VPHADDDyou would first do twoAsUint32x8and then shift/pack somehow to get aUint32x8that you can thenReduceAddto the desired result.Really the arg is
Uint2x4, as the permutation indexes are 0-3. Not sure whether it is necessary to enforce that or not. Similarly for the shift/rotate operations. Maybe we just handle the input restrictions in comments.This should really generate a
Uint64x4, and be calledMulWiden. Then a subselect(Uint64x4).Evens() Uint64x2afterwards would together make aVPMULUDQ.Am I allowed to write
Uint32x4{99,0,0,0}.Broadcast()? I presume so. Might be clearer to doBroadcastUint32x4(x uint32) Uint32x4, although maybe we would be starting with simd types a fair amount. Maybe both?apparentlymart commentedon May 19, 2025
Thanks for writing this up! This seems like a good general direction; I like the distinction between low-level intrinsics and higher-level operations.
With that said, I find the "best-effort portability" angle a little concerning. You made analogy to the
syscallvs.osdistinction, where a similar concerning situation exists:syscallfunctions are largely named after POSIX functions even though on some operating systems they do something more complex than just directly wrapping the POSIX function of the same name.syscallare supported on all Go targets. For those that are supported on multiple operating systems, the functionality is nonetheless often subtly different in behavior, in performance, or both.pkg.go.devlets you choose between different platform-specific views of the docs for that package1, but it's hard to quickly understand which subset of the functionality is portable and which isn't.These and other concerns led to splitting
syscallinto a number of OS-specific packages, each of which is usable only on the platform it corresponds to. This then means that it's explicitly the caller's responsibility to use conditional compilation to call into the appropriate package for the target OS, rather than the low-level package attempting to provide that abstraction itself.Based on that experience, I would personally have expected a separate package of SIMD intrinsics per architecture, and then have the caller (either end-user code directly, or the higher-level portable wrapper implementation) use conditional compilation to address each implementation they intend to support.
I acknowledge that the proposal described the situation as "best-effort portability", and that this comment is essentially arguing for making no effort whatsoever to achieve portability at this initial level of abstraction. I think it's clearer to have one layer that is explicitly not portable and then a separate layer on top that provides portable wrappers for the subset that all targets have in common, since then it's easier to understand what is portable and what isn't.
I am essentially making an SIMD-architecture-flavored variation of the following from the syscall-splitting proposal:
Footnotes
I notice also that
pkg.go.dev's selections are currently focused mainly on OS differences, so if this proposal introduces a new package that differs significantly betweenGOARCHvalues then that would probably call for expanding pkgsite to support selecting between multiple CPU architectures, too. ↩ianlancetaylor commentedon May 19, 2025
This is the kind of decision that we made (following Inferno) with the assembler: we have the same instruction names across architectures, rather than following the conventions of other assemblers. I think it's a mistake. It is better for people who are only superficially familiar with the architecture. But it is worse for experts. Experts in effect have to write their SIMD code using the operations with which they are familiar, and then look up the translation into the names used by Go.
I also note that Intel architectures have some 400 SIMD instructions. Even granting that Go doesn't expect to support all of them directly (after all, you can always use the assembler), it is implausible to think that there are good and meaningful names for 200 different vector instructions.
Just as with the assembler oddball instructions wind up using the names that are standard for the architecture, the simd package will wind up doing the same. Then we have a mix of meaningful names and architecture-specific names, so nobody is really clear on what is going on. And Go is living on a separate island, which is absolutely fine when there is a good reason to do so, but what is the good reason in this case?
I think it's better to provide the names that experts expect, either in the form of the instruction names or in the form of the intrinsic functions that architectures defined for C/C++ programmers.
Once we have that it would certainly be fine to define another package that provides readable names for common operations, providing a modicum of architecture portability without attempting to support all operations.
cespare commentedon May 20, 2025
@ianlancetaylor writes:
I agree, but I think it's even worse than this, because I'm not sure it helps the "superficially familiar" newbie either. I had this experience a decade ago doing a bunch of Go assembly work as a non-expert. I was cross-referencing the intel manual, agner fog, objdump output, and so forth, and figuring out the Go assembler naming scheme just added an extra layer of learning that I had to do on top of it all.
(That said, I don't feel too strongly about this because what's proposed here -- where each method has a comment that clearly lists the corresponding amd64 instruction mnemonic -- seems much easier to figure out than the Go assembler.)
dr2chase commentedon May 20, 2025
The problem with those types is, what do you do with them? They're not valid operands to any other instruction, there's no operations on those partial SIMD registers (except when a truncated ZMM or YMM happens to fit in an XMM). The obvious next step is to widen them to the zero-extended wider type, which is what the operation produced anyway. So, skip all that.
The exception to this is if their destination is memory, and given the plan to obtain that by optimizing away the stores, we would need obvious-to-the-optimizer sub-vector store operations, if we don't support these weird small types.
randall77 commentedon May 20, 2025
I'd rather do
Even if the
Uint16x4is stored in the low part of a larger register.If you really want the results in the low half of a larger thing,
or something.
cherrymui commentedon May 20, 2025
@dominikh
In the proposal draft I actually had a section about constant operand, and I decided not to include it in the post 😂
Here it is:
Constant operands
Some machine instructions require a constant operand, or have a form with a constant operand. For example, on AMD64, getting or setting a specific element of a vector (
VPEXTRD/VPINSRDinstruction) requires a constant index. For shifts, one form is shifting all the elements by the same amount, which has to be a constant (the other form being shifting a variable amount for each element). Namely,It is recommended (and will be documented) that these methods are called with constant arguments, so it can generate efficient code. What happens if it is not? The corresponding C intrinsics may cause a compilation failure. We could choose to do an emulation, or generate a table switch (as the range of the constant operand is usually small), as a fallback path.
cherrymui commentedon May 20, 2025
@ianlancetaylor @cespare One reason to choose names that are more descriptive is that I think it makes the code much easier to read: General readers of the code could easily understand (or guess) what
Uint32x4.Adddoes, but not quite so for a function namedVPADDDwithout reading the Intel manual or looking up on the internet. I think generally for some code, there will be more users who read the code than who write the code. So I'm leaning towards on the reader side.For SIMD code writers, the methods will be clearly documented with the machine instruction names. So code writers can easily look them up. This is where it can be different from the assembler, for which we don't always have good documentations for the mapping. Another difference is that assembly code is hard to read in general, even if we try to make it more unified.
cherrymui commentedon May 20, 2025
For naming, a reference (I should have added a reference to the proposal text) is that C# seems to choose to use common, descriptive names for their vector intrinsics. E.g. the X86 and ARM intrinsics classes both define
Addfunctions to add two vectors (https://learn.microsoft.com/en-us/dotnet/api/system.runtime.intrinsics.x86.avx?view=net-8.0, https://learn.microsoft.com/en-us/dotnet/api/system.runtime.intrinsics.arm.advsimd?view=net-8.0). One difference is that they have different classes for different architectures and CPU features. Nevertheless, the names are common.cherrymui commentedon May 20, 2025
@randall77
As @dr2chase mentioned, the reason for that is that the smallest vector size is 128-bit. Perhaps we should name them
TruncateToUint16PackedLowor so to be clearer.I think it would be infeasible to define many types for "small" vectors. We can document the semantics for Permute with Uint32x4. We probably don't want to enforce that high bits are not set. It wouldn't be surprising that one could come up with clever tricks with a vector of indices with the right low bits but possibly nonzero high bits (e.g. using -1 to represent the last element).
We can choose a different name. I'm not a fan of the semantics either. But at this low-level package I'm leaning towards doing what the machine instruction does. At least AMD64 (the
VPHADDxinstruction) and ARM64 (theADDPinstruction) both do that...balasanjay commentedon May 21, 2025
+1 to apparentlymart's comment above about having per-architecture packages (with implementations that just immediately panic on any other architecture).
Given the proposal says this: "It is expected that for the majority of use cases in data processing and AI infrastructure, it will be possible to write just with the high-level API, which achieves portability and performance", it seems a little messy to mix the portable and non-portable APIs in the same package. And if we do intend to separate them, I think we should save the nice package name of "simd" for the portable API.
(All that said, I'm a big fan of this proposal, kudos!)
dominikh commentedon May 21, 2025
I'll also note that
and
preclude writing low-level SIMD in files that don't have any build tags and using bits of SIMD in otherwise "normal" Go code by guarding uses of SIMD with static checks on architecture and dynamic checks on features. That is, we cannot have a single implementation of a function that uses scalar code, AMD64 assembly or ARM64 assembly, depending on which architecture we're compiling for and which CPU features were detected at runtime. At a minimum, we'll need a SIMD-free implementation and a SIMDful implementation, in two separate files, or our code cannot build for any architecture that isn't supported by the
simdpackage. And once we use any instructions or register sizes that are architecture-specific, we need >2 files, one per architecture.Given those limitations, we might as well import different packages per architecture, avoiding the need for finding similar-but-different names for functions in the case of
Having multiple packages would also make it easier to browse documentation and might aid code completion in gopls--as in, I will want to write ARM64 SIMD code while working on my x86 computer and would like auto completion to work well.
Yes, this would cause code duplication for trivial SIMD code that would be portable, but how often will such trivial code really be written? And couldn't it be written with the planned high-level abstraction instead?
216 remaining items
tmthrgd commentedon Jun 21, 2026
Is there a reason we're not getting the AES instructions (
AESDecryptLastRound,AESDecryptOneRound,AESEncryptLastRound,AESEncryptOneRound) for ARM64 in go1.27? ARM64 has all the required instructions as part ofFEAT_AES, though they may not map 1:1 to thesimd/archsimdAPI.balasanjay commentedon Jun 22, 2026
I believe I've found another miscompile.
To reproduce, please checkout this WIP change and run
GOEXPERIMENT=simd go test -run ^TestDecode$ encoding/base64.The relevant lines of code:
Snippet of output:
Or-ing together a non-empty bitmask and an empty bitmask should result in a non-empty bitmask. But it didn't.
(Apologies for the somewhat messy reproducer, I have not tried to shrink it very much)
================
EDIT: reduced the code significantly and posted it as a separate bug here: #80140.
dr2chase commentedon Jun 23, 2026
The reason is mostly that we were in a tearing hurry to get a consistent-ish API for three architectures, and to get Arm64 added, and we just ran out of time/bandwidth.
cherrymui commentedon Jun 23, 2026
Update: Last week we released Go 1.27 RC1, which includes the experimental support of the portable size-agnostic
simdpackage (#78902), an update of thearchisimdAPI for AMD64, and the inclusion ofarchsimdsupport for Wasm 128-bit fixed-width SIMD and ARM64 Neon. The AMD64 API is changed along the line of #78979 (comment) and #78979 (comment) (with minor adjustments). Note that this is a breaking change, so that code using Go 1.26 experimental archsimd API may not build with Go 1.27; please update the code accordingly. Some functions are left as deprecated, which will probably be removed in Go 1.28.Please try out the new experimental simd package, new supported platforms, and the updated API with Go 1.27 RC. Thank you!
mnightingale commentedon Jun 29, 2026
I've had a better chance to port some NEON code, a few things I noticed.
Firstly I have found the documentation and method names much clearer.
func (x Uint8x16) ConcatAddPairs(y Uint8x16) Uint8x16is missingvpaddq_u8/ADDPand the signed version.TBXis missing but looks it just missed the cut for 1.27, not a significant impact emulating it withTBLalthough hoping for multi-register variants in the future.vtstq_u8/CMTSTmissing but can emulate asa.And(b).NotEqual(archsimd.Uint8x16{})perhaps doesn't need a method and the compiler could recognise the pattern, if it doesn't already.vld1q_u8_x4/LD1 {Vt.16B - Vt4.16B},[Xn]is loading multiple structures being considered for the future, or could the compiler optimise loads/stores?I've been caught out by
ToMask()a few times, it does "not zero" meaning set, where asvbslq_u8 / IfElselets you just pass a vector and it'll bitwise 1 the mask.I guess I'm asking why does it need to change the value to 0/1, can there be a way to just use a
Uint8x16as it is or just reinterpret as a mask?Example is for NEON, but it's the same on AMD64.
Basically I have to add a
mask.And(archsimd.BroadcastUint8x16(1))C arm64
Go arm64
and 1 (correct) {0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1} ToMask() (wrong) {1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1} result {42,106,106,106,106,106,106,106,106,106,106,106,106,106,106,106}amusman commentedon Jun 30, 2026
For the ToMask() case: the lane-boolean semantics are intentional for comparison-produced masks, but as you've found they're the wrong tool for a raw bitwise select. I think we might need to add rules to
fold something like
(x&m)|(y&~m)intoBIF/BIT/BSLand have some helper like:cherrymui commentedon Jun 30, 2026
The Mask types are "opaque" types. It may or may not be the same bit representation as a vector. Conceptually it is a vector of booleans. When converting an integer vector to a mask (
IntMxN.ToMask), we interpret 0 as the falsy value and non-zero as the truthy value.It is probably possible to make the compiler recognize some patterns and optimize out the "compare-with-0" instruction in some cases.
+1 to @amusman that ToMask / IfElse is not the equivalent of bit selection.
Wasm has a BitSelect instruction. It seems reasonable to have a BitSelect API on other architectures.
es1928 commentedon Aug 5, 2026
I thought to share this article about performance impact of alignment (related to JDK hotspot): https://eme64.github.io/blog/2026/01/12/Alignment-Performance.html
The blog also contains other useful SIMD related articles specially about auto-vectorization which i hope is implemented someday in Go compiler, since there are many occasions of simple loops which greatly benefit from vectorization but using platform dependent (or even independent) intrinsics/api for them is not practical because of impact on code readability and maintenance.
Clement-Jean commentedon Aug 7, 2026
Is or will the interleaved loading of data be added for Neon? In C++, for example, you can do:
and this loads data differently from:
which is not interleaved.
This is interesting because you can load 64
uint8s, implement shifting (left and right by N) of vectors andmovemaskoperations (to bitmask/from_bitmask withuint64) more efficiently than non interleaved vectors.mnightingale commentedon Aug 7, 2026
Please can vpaddq_u8 / vpaddq_s8 / ADDP be added for Neon
Only the 16/32/64 versions are in RC2
qiulaidongfeng commentedon Aug 11, 2026
One example from the standard library: can we use compiler optimizations to avoid some manual rewrites that use archsimd, similar the kind of code rewrite done in CL 812960 when GOAMD=v3?
kolkov commentedon Aug 21, 2026
We prototyped an
archsimdrewrite of the Teddy SIMD prefilter in coregex (regex engine for Go, same algorithm as Rust's regex crate). On AMD64,Mask8x16.ToBits()compiles to VPMOVMSKB as expected and is essential for extracting match candidates from the nibble-lookup result.Is there a plan to add
ToBits()to ARM64 NEON and Wasm Mask types? ARM64 doesn't have a single PMOVMSKB equivalent, but the standard idiom (SHRN narrowing + UMOV, or SRI + pairwise add) is well-established — simdjson, Rust's memchr crate, and Vectorscan (Hyperscan ARM port) all implement it on NEON. Wasm has nativei8x16.bitmask.This is currently the main gap preventing portable
archsimdcode for byte-matching algorithms. We're targeting ARM64 parity with Rust's regex crate — coregex already beats Rust on 6 patterns on AMD64, but ARM64 SIMD is blocked by the missing mask extraction. We also maintain Born (Go ML framework, NEON GEMM shipped in v0.9.23) where bitmask extraction is a building block for planned quantization (packed weight unpacking), MoE expert routing, and sparse attention — all on the roadmap and all needToBits()on ARM64.Related: @Clement-Jean's question above about movemask on NEON.
gopherbot commentedon Aug 21, 2026
Change https://go.dev/cl/819140 mentions this issue:
simd, simd/archsimd: correct package documentation#128: draft the simd experience report (post-acceptance field report)
simd, simd/archsimd: correct package documentation