Should matrices be row-major or column-major?

[Qaanol]

Should 2-dimensional matrices be stored in row-major or column-major order?

There does not seem to be any consensus among languages and libraries:

Row-major order is used in C/C++/Objective-C (for C-style arrays), Mathematica, PL/I, Pascal, Speakeasy, SAS, and C#/CLI/.Net.

Column-major order is used in Fortran, OpenGL and OpenGL ES, MATLAB, GNU Octave, S-Plus, R, Julia, Rasdaman, Scilab.

Row-major order is the default in NumPy (for Python).

There is also the BLAS approach where operations take flags to indicate which matrices are transposed.

[Tub]

That depends. What data are you putting in there?

If a row of data is meaningful on its own (like in a database table), use row-major. If it's just a matrix that's only meaningful as a whole, it doesn't matter much.

If you're really worried about doing it wrong, consider fitting the memory model to your preferred iteration order. Traversing memory linearly increases cache locality, which can have an effect on your performance if your matrices are large. Also take a look at SIMD instructions, they only work on values in adjacent memory locations.

Bottom line, for matrix multiplication, you obviously store one of them in row-major and the other in column-major for maximum performance. That totally does not complicate things and will certainly not lead to bugs or other overhead that outweighs the gains from faster multiplication.

[Flumble]

I don't even know which is which, but since OpenGL wants column-major and graphics (or physics or whatever's up to 4 dimensions) are the only valid use of matrices, column-major is the right choice. No matter how counter-intuitive.

Bottom line, for matrix multiplication, you obviously store one of them in row-major and the other in column-major for maximum performance. That totally does not complicate things and will certainly not lead to bugs or other overhead that outweighs the gains from faster multiplication.

Totally agree. Though I hope that at the very least modern GPUs have a special (single cycle!) instruction for 4x4 matrix multiplication.

Is there any reason why this thread is not in Religious Wars?

[pogrmman]

I don't even know which is which, but since OpenGL wants column-major and graphics (or physics or whatever's up to 4 dimensions) are the only valid use of matrices, column-major is the right choice. No matter how counter-intuitive.

Graphics and physics certainly aren't the only valid uses of matrices. For instance, try implementing a neural net without matrices -- it's possible, but it's a total waste of memory and code for the vast majority of neural net architectures.

Matrices are great if you've got to store and process some serif values, which can come up in loads of different areas.

[EmelinaVollmering]

I does not matter. Most of the times, teacher use the row-major. And it is being followed from one generation to another. You can also use the column-major instead and make the analogues changing.

In my case, I will see, which way lower number of operations are performed. I will use row-major or column-major in that direction.