## Should matrices be row-major or column-major?

A place to discuss the implementation and style of computer programs.

Moderators: phlip, Moderators General, Prelates

Qaanol
The Cheshirest Catamount
Posts: 3058
Joined: Sat May 09, 2009 11:55 pm UTC

### Should matrices be row-major or column-major?

Should 2-dimensional matrices be stored in row-major or column-major order?

There does not seem to be any consensus among languages and libraries:

Wikipedia wrote:Row-major order is used in C/C++/Objective-C (for C-style arrays), Mathematica, PL/I, Pascal, Speakeasy, SAS, and C#/CLI/.Net.

Column-major order is used in Fortran, OpenGL and OpenGL ES, MATLAB, GNU Octave, S-Plus, R, Julia, Rasdaman, Scilab.

Row-major order is the default in NumPy (for Python).

There is also the BLAS approach where operations take flags to indicate which matrices are transposed.
wee free kings

Tub
Posts: 381
Joined: Wed Jul 27, 2011 3:13 pm UTC

### Re: Should matrices be row-major or column-major?

That depends. What data are you putting in there?

If a row of data is meaningful on its own (like in a database table), use row-major. If it's just a matrix that's only meaningful as a whole, it doesn't matter much.

If you're really worried about doing it wrong, consider fitting the memory model to your preferred iteration order. Traversing memory linearly increases cache locality, which can have an effect on your performance if your matrices are large. Also take a look at SIMD instructions, they only work on values in adjacent memory locations.

Bottom line, for matrix multiplication, you obviously store one of them in row-major and the other in column-major for maximum performance. That totally does not complicate things and will certainly not lead to bugs or other overhead that outweighs the gains from faster multiplication.

Flumble
Yes Man
Posts: 1998
Joined: Sun Aug 05, 2012 9:35 pm UTC

### Re: Should matrices be row-major or column-major?

I don't even know which is which, but since OpenGL wants column-major and graphics (or physics or whatever's up to 4 dimensions) are the only valid use of matrices, column-major is the right choice. No matter how counter-intuitive.

Tub wrote:Bottom line, for matrix multiplication, you obviously store one of them in row-major and the other in column-major for maximum performance. That totally does not complicate things and will certainly not lead to bugs or other overhead that outweighs the gains from faster multiplication.

Totally agree. Though I hope that at the very least modern GPUs have a special (single cycle!) instruction for 4x4 matrix multiplication.

Is there any reason why this thread is not in Religious Wars?

pogrmman
Posts: 514
Joined: Wed Jun 29, 2016 10:53 pm UTC
Location: Probably outside

### Re: Should matrices be row-major or column-major?

Flumble wrote:I don't even know which is which, but since OpenGL wants column-major and graphics (or physics or whatever's up to 4 dimensions) are the only valid use of matrices, column-major is the right choice. No matter how counter-intuitive.

Graphics and physics certainly aren't the only valid uses of matrices. For instance, try implementing a neural net without matrices -- it's possible, but it's a total waste of memory and code for the vast majority of neural net architectures.

Matrices are great if you've got to store and process some serif values, which can come up in loads of different areas.

EmelinaVollmering
Posts: 11
Joined: Tue Oct 18, 2016 10:24 am UTC

### Re: Should matrices be row-major or column-major?

I does not matter. Most of the times, teacher use the row-major. And it is being followed from one generation to another. You can also use the column-major instead and make the analogues changing.

In my case, I will see, which way lower number of operations are performed. I will use row-major or column-major in that direction.